Data Engineering - Data, Lakehouse and AI Data Platform Engineer - Analyst - New York
Tasks
- Build batch and streaming data pipelines
- Collaborate with data consumers and platform teams
- Develop raw refined and curated datasets
- Implement testing monitoring and issue resolution standards
- Improve pipeline reliability performance and maintainability
- Model data for business entities and historical change
- Optimize SQL queries and data processing
- Troubleshoot pipeline failures
- Validate data quality and reconciliation
Perks/Benefits
- N/A
Skills/Tech-stack
ANSI SQL | Apache Iceberg | Apache Spark | Avro | CI/CD | Databricks | Hadoop | JSON | Java | Kafka | Kubernetes | Parquet | Python | Release Management | SQL | Snowflake | Testing | Version control
Education
Roles
Regions
Countries
States
Cities
Related jobs
-
Solution Architect – Microsoft Azure Data & Fabric USD 150K-180KApache Spark | Azure Blob | Azure Blob Storage | Azure Data | Azure Data FactorySenior-level ContractSeattle3h ago
-
SYSTEM ENGINEER - Computer Network Support - AI/ML - 6+ yrs of Experience - TS/SCI w/Poly clearance is required - ES A USD 148K-152KAgile Development | Confluence | Jira | LLM | Process Engineering401k match | Dental insurance | Disability insurance | Life insurance | Medical insuranceMid-level Full TimeFort George G Meade, United States3h ago
-
AWS | AWS Glue | AWS Lambda | Amazon Redshift | Amazon S3Senior-level ContractLos Angeles, United States3h ago
-
Sr.AI ML Developer (Python) USD 115KAgentic AI | Agile | Fine Tuning | Generative AI | Language ModelsSenior-level Full TimeUS-FL-Tampa4h ago
-
Sr.AI ML Developer (Python) USD 110KAgentic AI | Fine Tuning | Generative AI | Language Models | Large Language ModelsSenior-level Full TimeUS-FL-Tampa4h ago
-
C++ | Cloud Storage | Data Analysis | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA; Seattle, WA, USA6h ago
-
Senior Software Engineer, Data USD 126K-189KAPI Development | AWS | Airflow | Docker | ETL401k plan | Commuting Stipend | Dental insurance | Dependent Care Flexible Spending Account | Employee assistance programSenior-level Full TimeSeattle, WA12h ago
-
Senior-level Full TimeNew York16h ago
-
Staff software engineer, data products USD 202K-255KBackend Development | Data Integrity | Data Modeling | Data Pipelines | Data TransformationIn office 4 days per week | Remote-friendlySenior-level Full TimeSan Francisco16h ago
-
Agentic AI Developer USD 140K-200KAgent Framework | Asynchronous programming | Autonomous Agents | Azure Cognitive | Azure Cognitive SearchMid-level Full TimeSan Francisco, CA, United States16h ago
-
Data Mining | Data labeling | Dataset creation | Deep learning | Knowledge DistillationSenior-level Full TimeFoster City, CA17h ago
-
Senior Data Engineer USD 82K-193KAgile | BigQuery | CI/CD | Data Engineering | Data PipelinesCompany-Paid Holidays | Dental insurance | Disability insurance | Employee assistance program | Life insuranceSenior-level Full TimeBridgewater, NJ, US17h ago
-
Senior-level Full TimeNashville, Tennessee, United States17h ago
-
Senior Data Engineer USD 155K-190KAmazon Redshift | Apache Airflow | Dagster | Data Compliance | Data GovernanceSenior-level Full TimeNortheast17h ago
-
Senior Data Engineer USD 155K-190KAmazon Redshift | Apache Airflow | Dagster | Data Governance | Data QualitySenior-level Full TimeSan Francisco17h ago
-
Machine Learning Engineer - ETA Team USD 137K-299KAirflow | Apache Spark | Deep learning | Experimentation | Feature Engineering401k plan employer matching | Basic life insurance | Commuter benefits match | Dental insurance | Disability insuranceMid-level Full TimeSunnyvale, CA; San Francisco, CA; Seattle, …18h ago
-
Analytical Engineer USD 60K-80KDAX | Data Governance | Data Modeling | Data Streaming | Data ValidationMid-level Full TimeLancaster, PA, United States18h ago
-
Analytics Engineer, Sentry USD 146K-194KDashboarding | Data Engineering | Data Modeling | Data Pipelines | Data QualityCompetitive benefits | Health insurance | Paid time offMid-level Full TimeIrvine, California, United States18h ago
-
Manufacturing Engineer, Analytics USD 129K-171KLean Manufacturing | Microsoft Excel | PLSQL | PowerBI | PythonHealth benefits | Recovery BenefitsMid-level Full TimeCosta Mesa, California, United States18h ago
-
Senior-level Full TimeSeattle, Washington, United States18h ago
-
Senior-level Full TimeRedmond, WA, US19h ago
-
Sr Engineer II, Machine Learning USD 139K-183KAWS | Bioinformatics | Cell analysis | Cloud Computing | Computer Vision401k match | Discretionary bonus | Flexible paid time off | Health insurance | Stock optionsSenior-level Full TimeSan Diego - Headquarters19h ago
-
Data Engineer USD 84K-110KAPI Development | AWS | Apache Airflow | Apache Spark | AzureHybrid work scheduleEntry-level Full TimeSan Diego, California19h ago
-
Senior Principal AI/ML Engineer - AI Program USD 172K-225KCI/CD | Cloud Computing | Computer Vision | Data Engineering | Data ScienceSenior-level Full TimeRochester, MN, United States19h ago
-
Machine Learning and AI Developer USD 85K-192KAI experiments | Agent Builder | Agentic AI | Cloud Orchestration | Cloud platformAdoption and surrogacy expense reimbursement | Community service time off | Employee resource groups | Fertility treatments | Flexible family care daysMid-level Full TimeDearborn, MI, United States19h ago