The task

The development and testing of powertrains is facing enormous challenges due to the explosive growth in data rates, higher demands on innovation and performance and stricter legal requirements. These should be countered with data management based on big data.

The challenge

Big data technology is already being successfully tested in many places for the evaluation of large amounts of data from the powertrain. The next step, the use of big data in productive systems, presents users with completely new challenges, as this requires integration with existing applications, security systems, provision for the end user, and much more.

our solution

In this project, a big data environment based on Hadoop was designed and a Hadoop cluster was installed and put into operation. The cluster was connected to a continuous data source for the analysis of measurement data via a data loading path. Big data workflows convert the measurement data into an analysis format and check the data quality before it is made available for analysis. By using scalable databases, data can be localized in seconds using metadata and bundled for analysis. Using a specially developed high-level analysis language for time series, engineers create complex data analyzes on big data themselves in a short amount of time and use an associated SDK to put them productively as apps.

The customer benefit

The cluster users were able to use the cluster with modern methods of data science in several workshops. For this purpose, analyzes were implemented on Python / Spark, including a search for patterns in measurement channels, several extensive reports on an endurance vehicle fleet (e.g., endurance report, cold start report, tank report, misfire report), test bench and quality assurance evaluations, as well as jerk detection using spectral analyzes and machines Learning / deep learning classifiers.