Jobs at Central Business Solutions, Inc

View all jobs

Sr. Developer/Technical Lead

Parsippany, NJ
Position Role/Tile: Sr. Developer/Technical Lead
Location: Parsippany, NJ.




Responsibilities:


Project is migration of a SQL MI Datawarehouse's on-prem ETLs from SAP Data Services to Azure Data Factory / Databricks

Against the 5 Technical Skill areas below, we need candidates that score 3-4 across several/most of the areas. We are not expecting to find perfect, we can create a blended team to cover all the bases. But we do need senior / experienced people.

Scoring:
  1. Some awareness / training / POC – but never implemented
  2. A small amount of experience implementing
  3. 3-6 months of experience implementing / good understanding
  4. 6+ months experience implementing / excellent understanding

Technical Skills:
  1. General Azure and DevOps Skills
    1. Understand overall Azure Data Services – Data Factory, Data Lake, Databricks, and SQL MI, DevOps
  2. Azure Data Factory
    1. Install, Configure, Scale, and Optimize Azure Data Factory the self-hosted integration runtime.
    2. Develop Azure Data Factory Pipelines for target data warehouse tables (dims, facts, associations, references, and audits)
    3. Develop Azure Data Factory Pipelines for Daily (Intraday) and Nightly batch processes
    4. Configure Data Factory Log Analytics and produce reporting of Azure Data Factory pipeline performance
    5. Monitor Data Factory Logs Analytics to detect degrading pipeline execution performance
    6. Troubleshoot and Optimize Azure Data Factory pipeline performance
  3. Azure Data Lake Storage Gen2
    1. Understand and apply best practices and patterns for organizing the data lake and datasets
    2. Understand and apply best practices and patterns for partitioning and storing datasets based on execution patterns and rate of change in datasets
    3. Understand and apply best patterns and practices for dataset archival, retention, and recovery
  4. Azure Databricks
    1. Develop ETL notebooks reading extracted raw data from Data Lake, transforming data using Spark SQL and Python, and loading processed data into Delta Lake and SQL Managed Instance
    2. Understand the apply best practices for creating and maintaining Delta Lake objects
    3. Understand and apply best practices for maintaining Delta Lake CDC datasets
    4. Understand and apply best practices for making connections from Databricks to Azure Data Lake and SQL Managed Instance
    5. Understand and apply best practices for optimizing Databricks execution performance including changes to Databricks infra and or sql/python code
  5. SQL Managed Instance
    1. Understand SQL Managed Instance capabilities and limitations
    2. Understand and apply best practices for loading SQL MI Staging tables from Databricks
    3. Troubleshoot and Optimize SQL MI Performance


Central Business Solutions, Inc,
37600 Central Ct.
Suite #214
Newark, CA 94560.
Central Business Solutions, Inc(A Certified Minority Owned Organization)
Checkout our excellent assessment tool: http://www.skillexam.com/
Checkout our job board : http://www.job-360.net/
=====================================================
Central Business Solutions, Inc
37600 Central Court Suite 214 Newark CA, 94560
Phone: (833)247-8800 Fax: (510)-740-3677
Web: http://www.cbsinfosys.com
=====================================================
Powered by