Job description
Role: Data Engineer
Location: Atlanta, GA
FULL-TIME(W2)
JOB DESCRIPTION:-
Mandatory SKILLS
- PySpark programming
- Databricks knowledge
- Pipeline creation using Databricks
- ETL knowledge
- Data Analytics knowledge
- Familiar with MS Azure
MUST HAVE
- Designing and implementing highly performance data ingestion & transformation pipelines from multiple sources using Databricks and Spark
- Streaming and Batch processes in Databricks
- Developing scalable and re-usable frameworks for ingestion and transformation of large data sets
- Data quality system and process design and implementation.
- Integrating the end to end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times
- SPARK performance\tuning\optimisation
- Have working experience in MS Azure
GOOD TO HAVE
- Providing technical guidance for complex geospatial problems and spark data frames
- Working with other members of the project team to support delivery of additional project components (Reporting tools, API interfaces, Search)
- Evaluating the performance and applicability of multiple tools against customer requirements
- Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints.
Job Type: Full-time
Pay: $115,000.00 - $130,000.00 per year
Benefits:
- 401(k)
- Health insurance
Experience level:
- 10 years
- 7 years
- 8 years
- 9 years
Schedule:
- 8 hour shift
Ability to commute/relocate:
- Atlanta, GA: Reliably commute or planning to relocate before starting work (Required)
Experience:
- Data Engineering: 7 years (Required)
- Databricks: 4 years (Required)
- Spark: 4 years (Preferred)
Work Location: In person
caravetterealestate.com is the go-to platform for job seekers looking for the best job postings from around the web. With a focus on quality, the platform guarantees that all job postings are from reliable sources and are up-to-date. It also offers a variety of tools to help users find the perfect job for them, such as searching by location and filtering by industry. Furthermore, caravetterealestate.com provides helpful resources like resume tips and career advice to give job seekers an edge in their search. With its commitment to quality and user-friendliness, caravetterealestate.com is the ideal place to find your next job.