What is Data Engineering

What is Data Engineering

Goal Of Data Engineer

  • Make robust, scalable and quality data available for fact-finding and data-driven decision
  • Doing data manipulation with the help of tools and technologies

Data Engineer Involves in


Collecting Data Involves

  • Collecting data involves extracting, integrating and organising data from different sources
  • Collecting data from databases, csv files, data lakes, data warehouse.

Processing Data Involves

  • Cleaning, transforming and preparing data to make it usable
  • Pipelines for extracting, transforming and loading data

Storing Data Involves

  • Storing data for easy availability of data
  • Make data scalable for more availability
  • Ensuring data privacy, security, compliance, backups, monitoring and recovering

Making Data Available Involves

  • Data available by creating Api's, services and some programs to retrieve data and pass to end user
  • Checks and balance to ensure data security