Description:
In this role, you will be responsible for designing and implementing DataBricks-based solutions to harness the power of data for analytics, reporting, and machine learning.
What you will do:
- Designing: Collaborate with stakeholders to create scalable and efficient data architecture solutions using DataBricks.
- Alignment: Ensure that the architectural designs align with the organization's business objectives and meet the data processing requirements.
- Lead Development: Take the lead in designing and developing data pipelines within the DataBricks environment.
- ETL Optimization: Ensure that data ingestion, transformation, and loading (ETL) processes are optimized for performance and reliability.
- Data Storage: Architect data lake and data warehousing solutions using DataBricks, focusing on best practices for data storage, partitioning, and efficient querying
What you'll need
- Proven Track Record: Demonstrated experience as a DataBricks Architect or similar role, with a strong background in data engineering, data architecture, and analytics.
- In-depth Knowledge: A deep understanding of DataBricks, including cluster management, notebook development, and automation using DataBricks APIs.
- Cloud Experience: Experience with cloud platforms such as Azure, AWS, or GCP, and their integration with DataBricks.
- Strong Communicator: Strong communication and collaboration skills to work effectively with cross-functional teams and convey technical concepts to non-technical stakeholders.