The German Fraunhofer Institute for Optronics, System and Technologies and Image Exploitation has presented its EnvP retrofit kit. Here’s the lowdown on the system. 

KEEPING IT BRIEF

  • The EnvP kit lets you operate machines remotely using the mobile phone network.
  • Commands are sent using the tractor’s CANbus.
  • A live video feed shows what the operator would see if they were sitting in the cab.

The Environment Perception Kit, or EnvP for short, is a retrofit package for tractors and mobile machinery providing them with environmental perception. The kit was developed by the “Embedded Intelligent Systems” department at the Fraunhofer Institute for Optronics, System Technologies and Image Exploitation (Fraunhofer ISOB-AST). It allows big kit, such as tractors, to be remotely operated from several hundred kilometres away.

When developing EnvP, the institute wasn’t even thinking about general farm work; the target was much more specialist tasks with real safety risks such as maintaining green spaces formerly used for military training. 

A box for the roof

The EnvP kit is all contained in a 83cm x 110cm x 32cm box that is designed to be mounted on the vehicle’s roof. Weighing 35kg, this box houses a dual GNSS receiver with an integrated inertial measurement unit (IMU).

The receiver processes satellite signals from two antennas. The IMU detects movement in all three directions to compensate for pitching and swaying from the cab as the vehicle is moving. 

The roof box also contains a mobile phone network modem and all of the computing hardware. Arms on the left and right of the box carry the antennas. For centimetre-level accuracy, the satellite signals are corrected via RTK.

The box also has a 3D lidar system and a 2D full HD camera, both looking forwards. The lidar sensor uses laser beams and more than 128 channels to scan the surroundings on multiple planes, enabling the sensor system to detect any obstacles. The camera records a high-resolution live video feed, which is transmitted via the phone network back to the control centre. From there, an operator drives and controls the machine using a joystick.

Only on drive-by-wire machines

Connecting the retrofit kit to a tractor or self-propelled machine only needs the two cables: one for power and one CANbus cable for data communication.

In order to use EnvP for remote operation, the host machine has to be drive-by-wire capable. This means it must have an electro-hydraulic steering valve. 

The electronic system of the vehicle also needs to be able to process the electronic commands for driving both forwards and backwards at different speeds.

At Agritechnica, Fraunhofer ISOB-AST demonstrated the feasibility of driving a tractor remotely. Using a remote control, we were able to operate the Claas Arion which was 200km away and navigate a test course. As the demo tractor still lacked the safety stop system, which is triggered by the lidar system when it detects an obstacle, there was someone sitting in the cab, and the tractor was only moving at walking pace. There was no rear implement on the tractor either, but you could still get a sense of what it will be like to operate the Arion remotely.

One operator can drive and control a tractor remotely using a joystick…

Virtual view from the cab

For the demo at the show, Fraunhofer ISOB-AST set up a private 5G campus network. Alternatively, a satellite network such as Starlink or OneWeb would also be suitable for sending large amounts of data quickly, explained Norbert Fränzel, group leader at Fraunhofer ISOB-AST.

Thanks to the live video, the operator can see the area in front of the vehicle as if they were sitting in the cab. However, lots of other sensory impressions are missing, such as the vibration of the tractor as it moves, or being able to hear changes in the engine speed.

…provided Fraunhofer ISOB-AST has installed the environmental perception sensor system on its roof, as shown here on this Claas Arion.

Apart from that, it also takes a bit of practice to drive a large tractor with a joystick. For example, we first had to get a feel for how the steering wheel responded. As a result, we didn’t always quite manage to guide the tractor neatly through some of the tighter bends on the first attempt. On top of that, we couldn’t reverse the tractor during the live demonstration. That manoeuvre would require a reversing camera and a rear-facing lidar scanner.

The high-res live image at the top left shows the view from the cab.

Not for retail

End customers cannot buy the EnvP retrofit kit. Fraunhofer ISOB-AST is only offering it to machine manufacturers, who can properly use it as a tool for developing full autonomous functions on their vehicles. In other words, it is currently a pre-development tool.

The EnvP kit is designed as a modular system with complete flexibility on how it is expanded and configured. Other camera and sensor systems can also be connected. Depending on the sensor specification, the price is between €30,000 and €50,000.

Anja Böhrnsen

For more up-to-date farming news click here and subscribe now to profi and save.