Position Role/Tile: Cloud Data Engineer
Location: Santa Clara, CA.
- Integral team member of our AI and Analytics team responsible for design and development of Big data solutions
• Partner with domain experts, product managers, analyst, and data scientists to develop Big Data pipelines in Hadoop or Google Cloud Platform
• Responsible for delivering data as a service framework from Google Cloud Platform
• Responsible for moving all legacy workloads to cloud platform
• Work with data scientist to build Client pipelines using heterogeneous sources and provide engineering services for data science applications
• Ensure automation through CI/CD across platforms both in cloud and on-premises
• Ability to research and assess open source technologies and components to recommend and integrate into the design and implementation
• Be the technical expert and mentor other team members on Big Data and Cloud Tech stacksYour Experience
• 5+ years of experience with Hadoop or Cloud Technologies
• Expert level building pipelines using Apache Beam or Spark
• Familiarity with core provider services from AWS, Azure or GCP, preferably having supported deployments on one or more of these platforms
• Experience with all aspects of DevOps (source control, continuous integration, deployments, etc.)
• You have experience with containerization and related technologies (e.g. Docker, Kubernetes)
• Experience in other open-sources like Druid, Elastic Search, Logstash etc is a plus
• Advanced knowledge of the Hadoop ecosystem and Big Data technologies
• Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Impala, Spark, Kafka, Kudu, Solr)
• Knowledge of agile(scrum) development methodology is a plus
• Strong development/automation skills
• Competent reading and writing Scala, Python or Java code.
• System level understanding - Data structures, algorithms, distributed storage & compute
• Can-do attitude on solving complex business problems, good interpersonal and teamwork skills
• Degree in Bachelor of Science in Computer Science or equivalent.
Central Business Solutions, Inc,
37600 Central Ct.
Newark, CA 94560.