Big Data Engineer

Big Data Engineer

Sciente Consulting Pte. Ltd
Kuala Lumpur
2 - 12 Years
Not Specified

Job Description

Mandatory Skill-set

  • Bachelor in IT, Computer Science or Engineering;
  • Hands on experience using Big Data technologies like Hadoop distribution (Hortonworks), Hive, HBase, Spark, Pig, Sqoop, Kafka and Spark Streaming.
  • Experience using Microsoft Azure technologies like Big Data Storage Gen2, Data Factory, Databricks, Azure ML;
  • At least 2-3 years of solid hands-on development experience with ETL tools to transform complex data structure in multiple data sources environment, such as Talend, SSIS and IBM DataStage;
  • At least 1-3 years of big data programming techniques on coding in Python, Java, R and Scala;
  • Experience in creating data model (relational and/or data warehouse), data mart design and implementation;
  • Strong knowledge in various database technologies (RDBMS, NoSQL and columnar);
  • Good understanding of data analytics and data visualization such as Qlik Sense, Tableau and Power BI;
  • Ability to communicate and present technical information in a clear and unambiguous manner;
  • Strong ability to work independently and cooperate with diverse teams in a multiple stakeholder’s environment;
  • Strong sense of work ownership, high affinity with anything data and a desire for constant improvements.
Desired Skill-set
  • Prior experience in Insurance domain.
Responsibilities

  • Design, develop, document and implement end-to-end data pipelines and data integration processes, both batch and real time;
  • Perform data analysis, data profiling, data cleansing, data lineage, data mapping and data transformation;
  • Develop ETL / ELT jobs and workflows, and deployment of data solutions;
  • Monitor, recommend, develop and implement ways to improve data quality including reliability, efficiency and cleanliness to optimize and fine tune processes;
  • Recommend, execute and deliver best practices in data management and data lifecycle processes, including modular development of data processes, coding and configuration standards, error handling and notification standards, auditing standards, and data archival standards;
  • Prepare test data, create and execute test plans, test cases and test scripts;
  • Collaborate with different stakeholders to understand data needs, gather requirements and implement data solutions to deliver business goals;
  • Provide technical support for any data issues and change requests, document all investigations, findings, recommendations and resolutions.

Job Details

Industry:

Function:

IT

Education:

Sciente Consulting Pte Ltd is ranked as Red Herring Top 100 Asia company during 2013 and ranked, 2nd year in-a-row, one of the Fastest Growing Technology Companies on the Deloitte Technology Fast 500 Asia Pacific 2011 and 2012.

Our success over the past years credits to the contribution by each and every associate, their innovative approach to add immense value to Clients, delighted Clients and doing business with uncompromising values.

Our vision is to build a global organization with a difference - which we shall achieve through our mission of 'delighted Clients & Community'.

Sciente Consulting is a specialist business technology consulting services firm focused in banking, financial markets, insurance & telecommunications industries serving large global multinational corporations.

Amalgamation of business & technology competencies, agile business model and our methodology of client alignment derives a unique Value Innovation which has been an integral part of the strategy to delight its Clients with value proposition and a clear differentiation.

Philanthropy through social innovations, is very much part of Sciente culture to build the corporation with a difference and to delight the community. MyTechLogy.com is one of the social innovations projects by Sciente.

Please visit www.sciente.com to know more about us and to view more career opportunities.

About Recruiter

349

Followers

144

Active jobs

Functions : IT

Industries : Other

Skills/Roles I hire for: IT

level Hiring For: Junior Level, Mid Level, High Level, Top Management

Similar Jobs

Career Advice to Find Better