If robots are ever going to be a daily part of our lives, they will need to react quickly and accurately to what is going on around them. That includes responding to what people do.
There is no mathematical algorithm that will tell a robot what to do in a social situation. The only “correct” answer is whatever is the accepted social norm.
Since robots are not actually intelligent in the human sense, the only way this can work is via data driven algorithms. In other words, we need to use the same mechanisms of Convolutional Neural Nets and Transformers that have led to the rise of ChatGPT, MidJourney and their various algorithmic cousins.
And like humans, those robots will need to respond immediately, without needing to stop and think. This will be particularly true in situations that involve physical contact with humans, such as accepting or handing someone a cup filled with hot coffee.
Which means that we are going to need to develop robots that have machine learning (ML) embedded in those parts of the robot which involve interaction with the physical world — such as robot hands and fingers.
Eventually we will figure out ways to gather human data to train such robotic ML algorithms. But that won’t happen in isolation. We will also need more sophiticated touch sensors, proximity sensors, and other equivalents of the fabulous sensing machinery built into every human body.
So this won’t happen right away. But after it does, it will all seem perfectly natural.