Help was needed fast; a team that could start immediately as the project deadline was extremely tight. We started work just six months before the product was due to be launched. As part of our IT Staff Augmentation service, we delegated colleagues to provide the resources and expertise needed to make the project a success.
We ventured into new territory, were able to start and learn quickly, and be flexible to meet any additional requirements that arose during the project.
Our collaboration with Universal Robots stems from one of our successfully completed projects last year. At the start of the project, it was important to secure resources quickly and efficiently from a remote location, as the client’s capacity was tight.
Providing the right conditions, we were able to start work more quickly than if the client had waited for recruitment or internal resources. As a result, the project was up and running within two weeks, resulting in significant time savings.
It was also important to be flexible in the allocation of resources during the assignment as needs and priorities were constantly changing. As a local partner to a global company, we acted as a solution provider, which further enhanced the success of the project.
The main objective was to add AI tools to Universal Robots’ robotic arms to make them as user-friendly as possible. It was important that end users could easily use the robot’s user interface and that developers could create new programs for the robots using AI technologies.
The collaborative work involved using AI to automatically calibrate the camera and recognise different objects, their position and orientation. Part of the learning process involved the robot recognising which object was in front of it and, if so, performing a response movement corresponding to that object.
Part of the ARTofINFO team made improvements to an existing part of the system, wrote new software for the robot and created a complex diagnostic interface to see what the camera was seeing and to give feedback on the recognition of different objects. The rest of the team carried out a full testing process.
In addition to developing the Robotic Operating System (ROS2) and the AI accelerator, we also integrated NVIDIA’s ISAAC ROS using AI models through ROS enabled object recognition (RT-DETR) and real-time segmentation (SAM2).
To store data in an efficient and user-friendly way, we designed a database schema (SQLite), which is an extension of the previous file-based storage. We also implemented ROS nodes and services to communicate with URCaps.
As part of the ROS diagnostics, we refactored the URCap dashboard, giving it a new look and feel and user experience. In the Calibration Studio, we enabled replay of robot motion and implemented new features.
In the Part Pose Detection URCap, we enabled users to create images and videos, annotate them, start teaching from the generated Coco file and test the trained model. The Dashboard URCap allows users to view current topics, nodes and features.
These innovations contributed significantly to the success of the project, improving the user experience and harnessing the power of AI.
We created a new end-to-end testing process for which we created a template document in Jira to be used in the future. We helped define a more robust and clear setup process and made the setup documentation more user friendly.
It was important to clarify what hardware was required for the AI Kit and we helped test new versions of PolyScope with the AI Kit. During testing we uncovered potential bugs at system, usage and hardware level and identified critical network issues.
Finally, we supported and assisted our beta partner, SZTAKI, in setting up the AI Kit to ensure successful testing and development.
Among the many other tasks, we would like to highlight the implementation of time synchronisation between the robot and the NVIDIA Jetson Orin, for which we used the NTP (chrony) tool. We also developed the ROS diagnostic system to provide real-time diagnostic measures for troubleshooting network and application problems between the robot and Orin.
We faced a number of challenges throughout the project, like the acquisition of domain knowledge, the lack of a working environment and the need to build everything from scratch, which required a significant investment of time and resources. The time difference caused further coordination difficulties between the teams (USA, Denmark, Hungary).
As this was an R&D project, new ideas were constantly being generated, which is natural in an innovation project. Testing could only be done on site as there was no robot available at the home office, which was an additional constraint.
As a team, we felt that we were also making great progress as individuals in our developmental attitudes. This was an international project, there was a big time difference in working with others and this required a different attitude from us as individuals.
We were able to work effectively in a distributed team, which included managing remote access, permissions and setting up the project infrastructure. We also learnt how to work effectively under pressure.
Despite the time difference, we were able to work very well with the teams, choosing the times when we could most effectively work together and find solutions to the problems we outlined. The flexibility of all team members was outstanding.
Throughout the project, everyone collaborated well together and gave each other real feedback, which not only facilitated the learning process but also improved the achieved results. This mutual support and cooperation created a positive atmosphere and contributed to the successful completion of the project.
We have also looked at the possibilities for future joint projects, with the aim of exploring and implementing low-code, no-code use cases with Universal Robots in the future.
