Testing: approach table -> rotate -> detect bottle -> grab it

Progress:

• grab the bottle and lift it a little
• move forward and rotates to detect the glass (use the front camera)
• move around the glass until it is in the desired spot (use the front camera)
• lift the bottle and pour liquid into the first glass
• lift a little bit more and pour liquid into the second glass
• descend and then let go of the bottle

Below is a video that shows how the robot approaches the table by respecting a certain distance, rotates around it and stops when it encounters the bottle within the center of the image.After front facing the bottle it goes forward until the bottle is detected, rotates until it is in the center of the image, and grabs it.

Image processing: table and bottle detection

Lately we've stepped into the computer vision area and worked on the image processing part of the project. The OpenCV library was used.

There are four main tasks that need image processing in real time:

  1. Table detection

      This is the first detection. The robot will localize the table from a distance and get close to it. 

  2. Bottle detection (1)

     The first bottle detection is done while rotating around the table, in order to localize the bottle's position. 

  3. Bottle detection (2) 

      The second detection is done while standing in front of the bottle. This is done mainly because of precision reasons.  

  4. Cup detection

     The last processing is done with the purpose of localizing the cup in which the liquid will be poured.

All of the above mentioned processing steps are done based on template matching, with various improvements within the algorithm.

What have we been up to?

      Up until now we've implemented steps 1 and 3, i.e., the table and bottle (2) detections.

      We have also worked on "centering" the captured image provided that we need this when the robot approaches the previously detected object.

     Apart from this, a LEGO ultrasonic sensor has been installed, which brought improvements to the grabbing of the bottle.