Demand for tech professionals has elevated after a short downturn following the beginning of the COVID-19 pandemic. And one tech job that has vacancies throughout the business is knowledge engineer. Accountable for creating the infrastructure vital to make use of, course of, and retailer giant quantities of data, knowledge engineers are seeing a rise not solely in job alternatives but in addition in instruments that assist they do their job.
Present knowledge engineering job market
A report launched June 29, 2022, by knowledge engineering supplier Dattell have a look at the present marketplace for this profession. To compile his findings, Dattell personally collected and analyzed giant quantities of information, matching particular applied sciences to job numbers.
With an eye fixed on the job marketplace for knowledge engineers, Dattell analyzed 340,000 completely different job postings. Of those, 35% are for positions in knowledge orchestration, 30% for knowledge storage, 29% for knowledge visualization, and 6% for knowledge processing.
Moreover, Dattell analyzed the languages utilized by knowledge engineers. Python is the most well-liked, most well-liked by 38% of employers, adopted by Java with 33% and SQL at 22%. Taken collectively, three of them account for 550,000 jobs.
UNDERSTAND: Recruitment Toolkit: Database Engineer (TechRepublic Premium)
Moreover, remuneration for knowledge engineering jobs varies from $60,000 as a beginning wage to as excessive as $180,000 for extra senior positions. Those that work with Kubernetes, Elasticsearch, PostgreSQL, and Terraform have the best salaries, with salaries starting from $140,000 or extra. And whereas there are many alternatives open to individuals working with Tableau and Energy BI, most of them supply lower than $100,000.
Information engineering expertise developments
In his analysis, Dattell compiled an inventory of the 20 hottest knowledge engineering applied sciences, pointing to instruments like Microsoft Energy BI, Terraform, Chef, Spark, Elasticsearch, Hadoop, and Kafka. The merchandise cited present help in areas together with knowledge storage, knowledge orchestration, knowledge processing, and knowledge visualization, indicating that there is no such thing as a phase of information engineering. dominate the remaining.
Based mostly on Dattell’s analysis, the 5 hottest knowledge engineering instruments are MongoDB, Tableau, Kubernetes, PostgreSQL, and Ansible, which cowl knowledge storage, knowledge orchestration, and knowledge visualization applied sciences.
Among the many instruments, Tableau and Kubernetes occupies the main place by way of the most important variety of jobs. Different instruments that create many open jobs are Ansible, Hadoop, Terraform, Splunk, Energy BI, MongoDB, and PostgreSQL.
And among the many knowledge orchestration instruments examined, Kubernetes is by far the chief, adopted by Ansible. Though each merchandise are in the identical area, every is used in a different way. Kubernetes permits professionals to handle and preserve container well being, whereas Ansible permits them to deploy configuration modifications and handle updates.
The rising reputation of free and open supply instruments
In response to Dattell, with so many free merchandise obtainable, employers don’t see the necessity to pay for knowledge processing instruments. In consequence, firms are in search of workers and consultants with experience in free and open supply applied sciences, akin to main instruments like Apache Spark and Apache Kafka.
For instance, with knowledge storage expertise, paid instruments are most well-liked by 59% of employers, whereas 39% nonetheless desire free instruments. Among the many hottest knowledge storage applied sciences analyzed by Dattell, MongoDB is the most well-liked, adopted by PostgreSQL, each are free. Some merchandise, akin to Elasticsearch, have each paid and free variations.