Big Data Logging

November 23, 2020

Efficient validation of e-mobility

The automotive industry is investing billions of euros in the development of new mobility concepts. Although experts are convinced that the widespread advancement and adoption of autonomous vehicles will soon be associated with the development of e-mobility, today´s development of electric (EV) and autonomous vehicles (AV) are running on two tracks. However, the advantages of such a combination are obvious: autonomous driving brings efficiency in driving and battery use, while EV technology drastically cuts down on emissions, fuel costs, and maintenance.

Due to the complexity of environmental and traffic conditions, the validation of such new mobility concepts is a big challenge for the automotive industry and its suppliers. The recording of data as a basis for the development of automated driving functions and autonomous driving easily exceeds conventional embedded storage solutions and capacities. For more than ten years, ViGEM has provided comprehensive, all-in-one solutions for collecting, storing, and transferring big data and has established itself as a technology leader in the field of Car Communication Analyzers (CCA).

01 ViGEM CCA9010
ViGEM data logger CCA 9010.
Next level of autonomous driving

Before merging onto roadways, self-driving cars will have to progress through 6 levels of driver assistance technology advancements. Most vehicles on our roads today belong to the category of level 0 or 1. This refers to cars that have systems allowing both machine and driver to share control (driver only or partly automated).. The use of ADAS functions helps pilot vehicles in level 2 (i.e., lane assist) and 3 (temporary hands off). The vision of the future is that the automation of driving reaches highly automated and autonomous driving technologies operating in level 4 and 5. For example, cars will communicate directly with other vehicles and infrastructure, such as parking lots or traffic lights. This new technology, called vehicle-to-everything (V2X), will be enabled by data from sensors and other sources that travel via high-bandwidth, high-reliability, and low-latency signals. This will not only improve safety but provide drivers and passengers with valuable information about the road ahead. Equipped with this kind of artificial intelligence, self-driving cars will be able to act as independent road users themselves.

02 ViGEM Infographic Big Data Logging ViGEM 2 1024x724
Mobile big data logger with interfaces for cameras, radar, lidar, and multiple other sensors

To realize the development of sophisticated, advanced driver assistance functions, a number of requirements for the authentic recording of all internal vehicle network traffic must first be fulfilled, e.g.:

  1. Logging of ground truth sensors and vehicle bus data for use in machine learning.
  2. Validation of new ADAS technology under real-world conditions to keep raw data for proof of correctness and to counter future legal challenges.
  3. Reproduction of realistic scenarios for Hardware-in-the-Loop (HIL) and Software-in-the-Loop (SIL) testing.
Challenge 1: Logging of ground truth data

The validation of complex functionalities often requires a huge number of kilometers/miles to be driven by fleets of test vehicles. The data acquired from these field operational tests (FOT) become valuable puzzle pieces in ascertaining the roadworthiness of a vehicle. Among the most complex functionality elements in a modern vehicle are ADAS features like adaptive cruise control (ACC), lane keeping, emergency braking, and of course, piloted driving. This is why accurate and high-performance data loggers are needed most during the development process.

03 ViGEM Infographic Increase BigData 1
Exponential increase of environmental data.

The validation of new ADAS functions requires a comprehensive collection of environmental and traffic data. New sensors, especially high-resolution cameras, as well as lidar and radar interfaces generate enormous amounts of data to supply the best “vision of the environment.” Capturing all such signals, a necessity in the development of automated driving systems, requires very high performance that goes beyond the typical embedded systems capability found in most traditional electronic control units (ECUs). For comparison: ten years ago, a fleet of cars generated a few hundred gigabytes of data per week. Today, a single vehicle in level-3 development easily generates 60 TB or more during one driving shift! These high data rates require recording capabilities of up to 10 Gbit/s per channel, and the aggregated data rate in cars can pass 50 Gbit/s in continuous operation.

“When installed in the trunks of test vehicles, data logger devices have to be very robust to guarantee reliable operation in automotive environments while enduring shocks and vibrations, as well as a wide range of temperatures. For example, the rugged ViGEM data loggers and removable data storages feature temperature stability ranging from usually -20 to +65° C”. explained Bernhard Kockoth, Advanced Development Lead, Vigem GmbH.

“The data loggers put all collected data onto one robust removable data storage with capacities of 16 TB, or up to 64 TB. These cartridges can be exchanged in a matter of seconds so that no valuable test-drive time gets lost. Removable data storage can be shipped from almost anywhere in the world to the data center where a CCA Copy Station performs a fast upload into the cloud while the vehicle keeps driving. A holistic approach to easy and reliable recording of ground truth data makes the CCA an ideal tool for field operational tests where a fleet of cars does not need to return to a location with data center upload capability after every test drive. In this manner, the read-out raw data is quickly available for further analysis purposes, such as use in machine learning or the development of artificial intelligence”, Bernhard continued.

Challenge 2: Limited electrical power

OEMs and suppliers of the EV industry face an additional challenge when advanced driver assistance systems (ADAS) and autonomous driving have to be tested on pure electrical platforms: the current draw from measurement equipment. In a combustion-powered vehicle, there is no problem in placing additional electronics and computer equipment consuming kilowatts of power. One simply adds an extra battery and an inverter.

But available power management in electric vehicles is different from that in combustion vehicles, and the mobile measurement equipment power draw is comparable to that of an air conditioning system. In some circumstances, drivers must deactivate their air conditioning in order to make it the remaining kilometers to the next charging station. The situation is similar for the power consumption of the measuring devices: lower power consumption is more important than high computational power. That´s why engineers need the best of both worlds for tests under realistic conditions: high computational ability running on low electrical power.

To deal with this challenge, ViGEM provides high performance data loggers that operate within low electrical power requirements. The compact Car Communication Analyzer CCA 9003 hardly requires more current than a car stereo and may be powered directly from most vehicle 12V circuits without the need for a second battery and power rail. It allows for longer operation time and better range when used in an EV.

04 ViGEM Logger In The Trunk
Mobile data logger in the trunk of a test vehicle.
Challenge 3: HiL testing

For the authentic reproduction of real-world scenarios, all captured data must be recorded with synchronous timestamps. ViGEM devices manage the distribution of time via the generalized Precision Time Protocol (gPTP). gPTP, defined in IEEE 802.1AS is a protocol used to synchronize clocks throughout a computer network, allowing clock accuracy in the nanoseconds range, making it suitable for measurement and control systems. It is used not only to synchronize multiple cascaded CCA devices but all connected capture modules in a distributed measurement setup, where incoming data packets get timestamped instantly at arrival. Timestamping the data with nanosecond resolution as close as possible to the sensor or ECU is essential for frame-by-frame scenario reconstruction in HIL labs.


The progress of automated driver assistance systems and autonomous driving functions will depend on qualification and validation processes. The upcoming age of mobility will combine these two emerging technologies: autonomous driving and fully electric cars. Advancements in the battery charge times, range, and reliability of electric vehicles can both define and accelerate the speed at which autonomous vehicles will be ready for real road trips.

Bernhard Kockoth

About the Author

Bernhard Kockoth is the Advanced Development Lead at ViGEM GmbH from Karlsruhe, Germany. Over the past 20 years, he has worked for a number of Tier-1s on automotive drivetrains, infotainment, ADAS, and since 2017 in measurement systems for automated driving. At ViGEM, he researches technology solutions to respond to future industry demands.

Our Latest Issue

Read & Download

2024 Spring EV News Magazine

Engage with the Pioneers & Innovators who are making the shift to electrified transportation a reality.


Check out my other articles

LiDAR Connectors

LiDAR Connectors

LiDAR Connectors AJ Born LiDAR (Light Detection and Ranging) systems are used in many applications for their ability to measure distances and create detailed 3D maps of environments accurately. From...