Sr. Hadoop/Big Data Developer

  • San Francisco, CA San Francisco California

  • Post Date: May 15, 2020

Key Skills: API, Apache Hadoop, Apache HBase, Apache Kafka, Apache NiFi, Apache Spark, Data Engineering, Data Acquisition, Data Analysis, Data Processing, Data Warehouse, Database, Debugging, Cloud, JSON, Java, Microsoft Windows Azure, Python, SQL, SaaS, Scala, Software, Software Deployment, Software Development, XML

  • Experience: 5 + Yrs
  • Domain: Information And Communication Technology
  • Job Type: Contract - Corp-to-Corp, Contract - Independent, Contract - W2

Job Description

• Partner on the design the next implementation of Client secure, global data and insight architecture, building new Stream processing capabilities and operationalizing “Unified Data Acquisition and Processing (UDAP) platform”
• Own medium-to-large size data engineering projects
• Discover, ingest, and incorporate new sources of real-time, streaming, batch, and API-based data into our platform to enhance the insights we get from running tests and expand the ways and properties on which we can test
• Assist business in utilizing data-driven insights to drive growth and transformation.
• Build and maintain data processing workflows feeding analytics domains.
• Facilitate reliable integrations with internal systems and third-party API's as needed.
• Support data analysts as needed, advising on data definitions and helping them derive meaning from complex datasets.
• Prototype new algorithms, experiment, evaluate and deliver actionable insights.
• Experiment with new tools to streamline the development, testing, deployment, and running of our data pipelines.
• Work with cross functional agile teams to drive projects through full development cycle.
• Help the team improve with the usage of data engineering best practices.
• Collaborate with other data engineering teams to improve the data engineering ecosystem and talent within Client.
• Creatively solve problems when facing constraints, whether it is the number of developers, quality or quantity of data, compute power, storage capacity or just time.
• Maintain awareness of relevant technical and product trends through self-learning/study, training classes and job shadowing.
All About You
• At least Bachelor's degree in Computer Science, Computer Engineering or Technology related field or equivalent work experience
• 5 to 10 years of experience in Data Warehouse related projects in product or service based organization
• Expertise in Data Engineering and implementing multiple end-to-end DW projects in Big Data environment
• Experience in building and deploying production level data driven applications and data processing workflows/pipelines
• Experience with application development frameworks (Java/Scala, Spring)
• Experience with data processing and storage frameworks like Hadoop, Spark, Kafka
• Experience implementing REST services with support for JSON, XML and other formats
• Experience with performance Tuning of Database Schemas, Databases, SQL, ETL Jobs, and related scripts
• Experience of working in Agile teams
• Strong analytical skills required for debugging production issues, providing root cause and implementing mitigation plan
Nice To Have
• Strong familiarity with batch processing and workflow tools such as NiFi
• Familiarity with modern BI and data exploration tools.
• Experience in developing integrated cloud applications with services like Azure or GCP
• Experience implementing machine learning systems at scale in Java, Scala, or Python and deliver analytics involving all phases like data ingestion, feature engineering, modeling, tuning, evaluating, monitoring and presenting

Share this vacancy

Related Jobs

Drools Architect

RichFactors, Renton, Washington

Key Skills Java, Webservices, Drools

Experience: 5 + Yrs

Sr. Full Stack Engineer, Elastic Search

Techvilla solutions, Pleasanton, California

Key Skills MONGO DB, MONGODB, NO SQL, PYTHON, MongoDB, AWS, Kubernetes, MongoDB, AWS, Mongo DB Open Source, Elasticache, NoSQL, Elastic Search, Java, ...

Experience: 5 + Yrs

Contact Longfinch

San Francisco, CA San Francisco California 94101