Data Lead Software Engineer – Python, Pyspark and AWS
New York, NY, United States
USD 177K-186K (estimate) Senior-level Full Time
Tasks
- Assess architectural designs for existing systems
- Automate recurring remediation tasks
- Build AWS based data workflows
- Design batch and streaming data pipelines
- Develop secure production code
- Drive AI-assisted engineering practices
- Establish automated testing standards
- Evaluate vendor and startup solutions
- Execute software design and development
- Improve code quality and delivery speed
- Review and debug code
- Support incident root cause analysis
- Troubleshoot production issues
Perks/Benefits
- N/A
Skills/Tech-stack
AWS Glue | AWS Lambda | AWS Step Functions | Agile | Amazon S3 | Apache Airflow | Apache Spark | Automated testing | CI/CD | Data Migration | Data Modeling | Data Warehousing | Databricks | Distributed Systems | Driven systems | Event Driven | Event Driven Systems | Git | Incident Management | Jenkins | Maven | NoSQL | Object-Oriented | Object-oriented programming | Peer review | PySpark | Python | Real Time | Real-time Data | Redshift | SQL | Secure Coding | Snowflake | Step Functions
Education
N/A
Roles
Engineer | Lead | Lead Software Engineer | Software Engineer
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R11d ago
-
Sr. Embedded Software/Firmware Engineer USD 180K-240KARM | ASPICE | AUTOSAR | C# | C++Competitive compensation | Dental insurance | Flexible hours | Medical insurance | Pre-IPO stock optionsSenior-level Full TimeUnited States - Remote R7h ago
-
Sr. Software Engineer, AI Infrastructure USD 144K-236KApache Beam | Apache Flink | Apache Spark | C# | C++Career development | Hybrid work | Stock grantsSenior-level Full TimeSunnyvale, CA, United States8h ago
-
Google Cloud Platform (GCP) Data Engineer USD 100K-203KApache Beam | BigQuery | Broadcast joins | CI/CD | CachingMid-level Full TimeArlington, VA9h ago
-
Google Cloud Platform (GCP) Data Engineer USD 100K-203KApache Beam | BigQuery | CI/CD | Cloud Storage | Cloud platformSecurity clearance providedMid-level Full TimeArlington, VA9h ago
-
API Design | Alerting | Data Modeling | Entitlement checks | Low LatencyHybrid work cultureSenior-level Full TimeHQ-San Francisco9h ago
-
API Design | Data Modeling | Low Latency | Low-Latency Systems | Rate LimitingHybrid work culture | In person meeting once per quarterSenior-level Full TimeHQ-San Francisco9h ago
-
API Design | Alerting | Data Modeling | Entitlement checks | Low LatencyHybrid work culture | In person once per quarter for remote rolesSenior-level Full TimeHQ-San Francisco9h ago
-
Lead Data Analyst, Product Analytics USD 150K-180KA/B | A/B Testing | Analytics engineering | B testing | BigQuerySenior-level Full TimeRemote, US R10h ago
-
Senior Data Engineer USD 144K-187KAWS Lambda | Amazon DynamoDB | Amazon ECS | Amazon EKS | Amazon S3401k employer contribution | Dental insurance | Flexible Remote Friendly Work Options | Health Savings Account employer contribution | Health insuranceSenior-level Full TimeAustin, Texas11h ago
-
Senior Data Engineer - Finance USD 172K-203KARR | AWS | Apache Iceberg | Apache Spark | CRMDental insurance | Employee discounts | Flexible wellness time off | Health insurance | Hybrid workSenior-level Full TimeHybrid - San Francisco, California R11h ago
-
Principal AI Engineer USD 130K-196KAI Safety | AWS | Agentic Workflows | Data Engineering | Evaluation401k match | Dental insurance | Disability insurance | Employee assistance program | Flexible spending accountSenior-level Full TimeDenver, CO12h ago
-
Sr. Data Engineer (HYBRID) USD 145K-160KAWS | AWS Glue | AWS Lambda | Amazon Athena | Amazon DynamoDBComplimentary club membership | Hybrid work schedule | Personal training | Pilates | Shop discountsSenior-level Full TimeNew York, NY, United States R12h ago
-
Staff Data Engineer - Data Engineering USD 207K-275KAccess Control | Apache Fluss | Apache Hudi | Apache Iceberg | Apache Paimon401k matching | Dental insurance | Disability insurance | Employee stock purchase program | Flexible PTOSenior-level Full TimeLivingston, NJ / New York, NY …12h ago
-
Cheminformatics | Cloud Computing | Data Analysis | Data Visualization | Deep learningSenior-level Full TimeRedwood City, California, United States12h ago
-
Senior MLOps Engineer I USD 170K-180KAWS | CI/CD | Canary deployments | Data Versioning | Deployment PipelinesAutonomy | Dental insurance | HSA Eligible | Inclusive culture | Medical insuranceSenior-level Full TimeSan Francisco, California, United States12h ago
-
Senior MLOps Engineer I USD 170K-180KAPI Integration | AWS | CI/CD | Canary Deployment | Computer VisionDental insurance | HSA options | Health insurance | Paid training | Stock options (eligibility)Senior-level Full TimeBoston, Massachusetts, United States12h ago
-
AI Developer USD 99K-198KAWS | Agentic AI | Azure | CI/CD | Data LakesHybrid work schedule | Leadership development | Professional developmentMid-level Full TimeWashington, DC12h ago
-
Software Engineer USD 85K-100KBiometrics | Computer Vision | Deep learning | Facial recognition | Image Geolocation401k match | Dental insurance | Disability insurance | Health insurance | Life insuranceMid-level Full TimeSomerville, Massachusetts, United States13h ago
-
Lead Data Modeler USD 130K-165KAWS Databricks | Azure DevOps | Business Data Analysis | Business data | Conceptual data modelingSenior-level Full TimeNew York, NY, United States13h ago
-
Lead Architect, GenAI Platform & Agentic AI USD 170K-220KAPI Gateway | API driven LLM | API-driven | AWS | AWS CDK401k with profit sharing | Disability coverage | Life insurance | Medical, prescription, dental and vision coverage | Paid HolidaysSenior-level Full TimeWoodbridge, NJ, United States13h ago
-
Senior-level Full TimeOrlando, FL, United States14h ago
-
AI Requirements Engineer USD 120K-145KAWS | Acceptance criteria | Agile | Analytics | Artificial IntelligenceMid-level Full TimeArlington, VA, United States14h ago
-
Algorithms | Concurrency | Data Structures | Database systems | Distributed SystemsCompany-matched 401k | Flexible schedule | Health coverage | Parental leave | Remote work flexibilitySenior-level Full TimeUnited States R14h ago
-
Data Analytics & Engineering Opportunities USD 65K-105KBusiness Intelligence | Microstrategy | Python | R | SQL401k match | Dental insurance | Disability insurance | Firm paid holidays | Health insuranceEntry-level Full TimeWashington, DC, United States14h ago