Astronomer Cloud Edition enables organizations to quickly adopt Apache Airflow, a code-based workflow management tool that is quickly becoming the industry standard for authoring, monitoring, and maintaining data pipelines.
CINCINNATI (PRWEB) March 14, 2018
Astronomer, the data engineering company, has launched an expansion to its Cloud Edition, a platform that collects, processes, and routes data whilst being entirely hosted in its own managed cloud environment. Astronomer Cloud Edition now enables organizations to quickly adopt Apache Airflow, a code-based workflow management tool that is quickly becoming the industry standard for authoring, monitoring, and maintaining data pipelines. Originally open sourced by Airbnb, Apache Airflow is currently being incubated by the Apache Foundation and has become a best-in-class tool for automating data workflows.
As organizations become more ambitious with their data, “standard” ETL solutions like drag and drop interfaces, standard schemas, and black-box tools can only do so much. These tools lack the level of flexibility and customization that companies need to leverage internal, market, and customer data as a competitive advantage. Apache Airflow solves this problem by providing engineers with the ability to quickly setup complex and contingency based data workflows connecting and transforming data across many different sources and silos.
Astronomer’s Managed Apache Airflow Module aims to accomplish 3 goals:
1. Work with data anywhere: Proliferation of SaaS silos and internal straddling of cloud/on-premise data environments means that data teams must be prepared to work with data across technical and corporate barriers.
2. Provide a world-class developer experience: Astronomer caters to the needs and desires of modern open-source developers with lightweight tools, a rich CLI and API, a locally mirrored dev environment and no requirement to use an inflexible and confusing GUI.
3. Minimize infrastructure issues: Secure deployment to a managed cloud environment and seamless horizontal scaling ensures that teams focus their time and effort on developing, improving and monitoring data workflows.
“Apache Airflow not only automates how and when your workflows run, but it’s also completely system agnostic, so it can fit into any organization’s stack.” said Greg Neiheisel, Astronomer’s CTO. “You can take any developer that knows Python, junior or senior, and they can use Airflow to write custom production-grade workflows. Our Cloud Edition makes Airflow into something an organization can start using immediately instead of having to deal with things like hosting or deploying.”
For organizations with stricter security needs, Astronomer Enterprise Edition offers a fully customizable version of the Cloud Edition, deployable to any cloud environment with a Kubernetes cluster.
The Rise of the Data Engineer
Software developers are now carrying the majority of the workload when it comes to data engineering—a collection of activities encompassing data collection, warehousing, cleaning and integration. This work typically accounts for 70 - 80% of the time spent generating insights from data, and is an ongoing responsibility as API’s, databases and data types constantly change.
"With Astronomer, data from our HR systems, database, communications platform, speech analytics tool and several other sources is automatically extracted, transformed and loaded into a data warehouse for analytics, which allows our data engineers to focus entirely on enriching our data sources and enhancing data modeling and reporting,” says James Cisneros, CIO of Dallas based, Real Time Resolutions.
Both Astronomer’s Cloud and Enterprise platforms enable developers and data scientists to automate much of this work through streaming and batch data collection and contingency-based workflow development through its Managed Apache Airflow module. Ultimately, Astronomer hopes to become a unified platform for all an organization’s data.
Astronomer is the automation and management platform for data engineering. Add a few snippets of code to your apps or sites to begin capturing data in Astronomer Clickstream - Cloud Edition, or deploy Enterprise Edition via Kubernetes to your cloud for maximum security and control. Then leverage Astronomer’s Managed Apache Airflow product to create and manage contingency-based data workflows for enhanced data tasks. Astronomer is trusted by companies ranging from startups to Fortune 100 around the world. For more information visit https://www.astronomer.io.