53 docs tagged with "Dagster supported"

Get utilities for ADLS2 and Blob Storage.

Orchestrate Airbyte connections and schedule syncs alongside upstream or downstream dependencies.

Orchestrate Airbyte connections and schedule syncs alongside upstream or downstream dependencies.

Easily integrate Dagster and Airflow.

This integration allows you to connect to AWS Athena and analyze data in Amazon S3 using standard SQL within your Dagster pipelines.

This integration allows you to send Dagster logs to AWS CloudWatch, enabling centralized logging and monitoring of your Dagster jobs.

This integration allows you to connect to AWS Elastic Container Registry (ECR), enabling you to manage your container images more effectively in your Dagster pipelines.

The AWS EMR integration allows you to seamlessly integrate AWS EMR into your Dagster pipelines for petabyte-scale data processing using open source tools like Apache Spark, Hive, Presto, and more.

The AWS Glue integration enables you to initiate AWS Glue jobs directly from Dagster, seamlessly pass parameters to your code, and stream logs and structured messages back into Dagster.

Using the AWS Lambda integration with Dagster, you can leverage serverless functions to execute external code in your pipelines.

Using this integration, you can seamlessly integrate AWS Redshift into your Dagster workflows, leveraging Redshifts data warehousing capabilities for your data pipelines.

The AWS S3 integration allows data engineers to easily read and write objects to the durable AWS S3 storage, enabling engineers to have a resilient storage layer when constructing their pipelines.

This integration allows you to manage, retrieve, and rotate credentials, API keys, and other secrets using AWS Secrets Manager.

The Dagster AWS Systems Manager (SSM) Parameter Store integration allows you to manage and retrieve parameters stored in AWS SSM Parameter Store directly within your Dagster pipelines.

Integrate Chroma vector database capabilities into your AI pipelines powered by Dagster.

The Databricks integration enables you to initiate Databricks jobs directly from Dagster, seamlessly pass parameters to your code, and stream logs and structured messages back into Dagster.

Publish metrics to Datadog from within Dagster ops and entralize your monitoring metrics.

Put your dbt transformations to work, directly from within Dagster.

Put your dbt transformations to work, directly from within Dagster.

Run dbt Cloud™ jobs as part of your data pipeline.

Integrate your pipelines into Delta Lake.

Easily ingest and replicate data between systems with dlt through Dagster.

Run runs external processes in docker containers directly from Dagster.

Read and write natively to DuckDB from Software Defined Assets.

Build ELT pipelines with Dagster through helpful asset decorators and resources

Integrate with GCP BigQuery.

Integrate with GCP Dataproc.

Integrate with GCP GCS.

Integrate Gemini calls into your Dagster pipelines, without breaking the bank.

Integrate with GitHub Apps and automate operations within your github repositories.

Dagstermill eliminates the tedious "productionization" of Jupyter notebooks.

Launch Kubernetes pods and execute external code directly from Dagster.

The Looker integration allows you to monitor your Looker project as assets in Dagster, along with other data assets.

Integrate OpenAI calls into your Dagster pipelines, without breaking the bank.

Centralize your monitoring with the dagster-pagerduty integration.

Implement validation on pandas DataFrames.

Generate Dagster Types from Pandera dataframe schemas.

Patito is a data validation framework for Polars, based on Pydantic.

Polars is a blazingly fast DataFrame library written in Rust with bindings for Python.

Represent your Power BI assets in Dagster.

Integrate with Prometheus via the prometheus_client library.

Represent your Sigma assets in Dagster.

Up your notification game and keep stakeholders in the loop.

Extract and load data from popular data sources to destinations with Sling through Dagster.

An integration with the Snowflake data warehouse. Read and write natively to Snowflake from Software Defined Assets.

Configure and run Spark jobs.

Establish encrypted connections to networked resources.

See and understand your data with Tableau through Dagster.

Integrate Twilio tasks into your data pipeline runs.

The TypeScript Pipes client allows integration between any TypeScript process and the Dagster orchestrator.

Using this integration, you can seamlessly integrate Weaviate into your Dagster workflows, leveraging Weaviates data warehousing capabilities for your data pipelines.

Orchestrate Airbyte Cloud connections and schedule syncs alongside upstream or downstream dependencies.

Orchestrate Fivetran connectors syncs with upstream or downstream dependencies.