I have known about the raspberry pi for quite some time. So why did I spend so much time working with embedded systems like pic and stm32 ? Initially I thought of the extremely low cost and power requirements for those devices. My initial estimates were for a controllable tracked video bot for as little as $25, which is still realistic.

I’m going to weigh the costs and benefits of each system and we can see they are different systems, kind of like comparing apples to oranges. So yes I think both systems are very desirable. The pi gets me where I want to be faster, and it can support ROS. To me ROS makes all the difference so I’m going down the Pi path for that reason. But I’m going to release all of my code for the stm and everything I have worked on so far in case someone else wants to still pursue the embedded path.

The last thing I completed was a phone controlled bot based off of the stm32f4 discovery board. I was using a bluetooth module and was getting about a 250×100 frame every 2 seconds on my phone. I have a full api thats easy to use and all of the source is open and free here. There are android and pure java clients that get the video feed. Even thought the HC-05 module boasts 1.3Mbps it also says 160Kbps which is the max speed I was able to push through it. If anyone figures out the actual max speed please let me know !

https://code.google.com/p/stm-phone-camera-bot/source/checkout

With all of this working at a frame every 2 seconds I still had in the back of my head that with jpeg that could be 1/16 compression giving me 8 frames per second. I was also planning to implement nordic rf 2401 which gives a possible 2 mbps which would greatly increase the frame rate. What I would end up with in the end would be pretty spectacular but it would not run ROS.

So lets compare everything.

Assuming !
Motor control circuit is the same
Power charge circuit is the same
And disregarding passive components as negligible

Plan A:

Stm32F4, Bluetooth module connect direct to telephone.
Difficulty to Implement: Moderate
Todo: Interface with Ov2460, more drivers for HC-05
Modular: Low

Plan B:

STM32F4 based, nordic chip. For this to work I would need to make a dongle board for converting the nordic input to usb. This would be very hard and possibly slow things down considerably, but at least I would need chips on both ends to control everything. I would use a lower cost stm32 for the Dongle end.
Development: Hard
Todo: dongle, spi nrf interface driver, camera driver

Plan C:

Use a raspberry pi (Model A) and usb camera, stream over wifi. Of course there are obvious issues with this like how do I embedd a usb dongle or webcam into the final product. In a few units I can dismantle and solder but I would need a better solution for mass production. Also I will need to integrate a usb hub whereas in the other designs that was not nessisary.
Difficulty: done
Todo: Add configuration, Create Circuit board, Test

Summarizing and including the model B since model A does not always seem to be available.

The big things to note are 150Mbps(pi) vs 1Mbps(STM) and the mA 1.5A (Pi) vs .180A (STM). So all things the same the STM plan would last 8 times as long as the Pi on the same battery charge. For example I’m using two CR123 Rechargables at 800mAh in the pi that gives me 32 Minutes, the STM would last for 4.3 hours on the same two light batteries. However when we are pumping motors continually the amps used by the PI might not be so important. For example if our motors (fa-130 for example) use from 200 to 2.2A each then we round that to 2A we need a larger battery solution. The 150Mbps vs 1Mbps is a no brainier if you are trying to push any decent video size through the device.

At the cost of only $14 more it seems like the Pi is the clear way to continue. The stm plans would only support one set of per devices like cameras, wireless etc whereas the pi has tons of drivers and a huge list of already supported devices.

So with all that in mind I’m moving forward knowing that I will be pulling something like 3A and will need a large battery solution. Since the motors and several other parts of the circuit would be the same regardless of the “brain” then we can always go back. But with things like ROS and Ubuntu I dont think there is going to be any turning back. The most I might do is use my STM32 knowledge to integrate smart perphs into the pi.

There are a plethora of different packages out there for streaming video. The one I found that is closest to real time is “mjpg_streamer” that does not mean its the best or that you cannot achieve better with another package, it just means this is the one that I got to work best so far.

To install follow the instructions here
http://www.tanzilli.com/video_streaming
If thats not enough info you can look here too
http://www.phillips321.co.uk/2012/11/05/raspberrypi-webcam-mjpg-stream-cctv/

The final command I used was a very low image size to give me about 10fps

mjpg_streamer -i "/usr/lib/input_uvc.so -d /dev/video0 -y -r 160x120 -f 10" -o "/usr/lib/output_http.so -p 8081 -w /home/pi/mjpg-streamer/mjpg-streamer/www/" -b

From there you can http into your pi on its ip at yourpiip:8081 and you should see pages to stream your video. The tricky thing that is not mentioned on those pages is the last -w argument is for the webpages which if you follow the instructions you delete. You should save those pages or when connecting you will get a “404 not found” because you pointed mjpg_streamer to host pages that dont exist. I have the full path in my argument as you see.

From there gpio was a snap I used WebIOPi you can find the instructions here
https://code.google.com/p/webiopi/wiki/INSTALL
and yes you can have them both running at once (camera gpio) as long as they are on different ports.

I installed both as a service so my pi starts right up and streams video and opens its gpio. So I hooked up some lights and put them in front of the camera. Now I can toggle them on and off from anywhere in the world (since I forwarded the pages) and see them.

The webiopi wont let me drive since I have to turn on two gpio at the same time to go forward or back, but it will be good for testing. Next I’m moving on to making my own javascript gpio control then I can connect it to the chassis and drive it around !

Though unfortunate events I burnt up my stm32f4 discovery, so I thought to myself now would be a good time to load up the raspberry pi and see what I could do with it.

I found an instructable with a link to Raspian with ROS built in. http://www.instructables.com/id/Raspberry-Pi-and-ROS-Robotic-Operating-System/step2/Writing-the-image-to-the-SD-card/

Then I found instructions on how to setup a wifi webcam that runs off solar batteries. This camera is hosted across the web in any browser. http://www.instructables.com/id/Raspberry-Pi-Completely-Wireless-IP-Camera-Solar/

To manipulate the gpio there are several different methods, there is a very exhaustive page here that shows how to do it in basically any language you want. Looks like pwm might be out of scope for the first run but controlling my already built motor drivers should be easy.http://elinux.org/Rpi_Low-level_peripherals

GOAL:

Putting this all together we should have what we wanted very quickly. A robot that can drive around using the wifi providing a video feed to a client. Looks like just by installing software the client will be a webpage. I will have to find a way to integrate the motor control and the video feed on the same page. If I could accomplish this then I would be far past where it has taken me about 2 years to get using MCU and integrating directly with the camera.

Once this works:

Daughter board
– Gets Power from batteries
– Charges Batteries
– Monitors Power
– Motor Drivers (4)
– Servo Drivers
– LED and Laser Connectors
– Camera Connector ?

Body
– Build a body that fits batteries, pi and daughterboard
– Construct and test

Other thoughts
I don’t know if we can take the schematic of the pi, strip out the stuff we dont want and include the stuff we do, such as wifi, motor drivers etc. If we could simply make our own version of the rPi that would be better than fitting a square peg in a hexagon hole.

There is tons of interest right now in the rPi, it is buzzing all over the internet and every Tom, Dick and Harry is making accessories for it. It will be nice to ride that wave instead of using MCUs that are less renown. I can say that if we make accessories they should cost less than the pi itself. If I make a full video bot package we should be able to keep the cost below $100.

Using the bluetooth modules I have become pretty familiar with them. There are several different models out there but most of them are banging the BC417 core for bluetooth.

The idea was we could have speeds up to the at commands for baud which has a max of  1382400 baud. With that we could do 3.5 frames a second with raw at 251×100 at 2 bpp, of course in a perfect world with no overhead. Then I would move onto a camera that has built in jpeg compression and get a 1/16 compression ratio moving my fps to 56 fps at which time I could start to adjust for a bigger image and get closer to 20fps but with a larger image.

Wrong ! After setting the bluetooth to a higher baud 921600 I maxed out the kbps at 160 rx. This does not make any sense to me so I look at the modules specs and I see

Rate: Asynchronous: 2.1Mbps(Max) / 160 kbps.

There was another spec for synchronous but usart is async. I really dont know why these are written this way, maybe its 2.1Mbps upload and 160kbps down or maybe its 2.1Mbps on the usart side and 160kbps max on the bluetooth side or maybe its simply showing that the max is 2.1Mbps and we need at least 160kbps for the other side.

So I dove deeper into the bc417 datasheet it shows the uart can communicate as fast as 2764800, but if you look at the rfcomm section 9.2.1 you see it has a 350kbps max data rate. I have no idea how any of these chips could have data coming in at a huge rate and going out at a much slower one, maybe I would see buffer overflow if I pushed to much. Actually I think I did at about 700+ bytes.

Bottom line max kbps is 350 for what we are using, all I can seem to get out of it is 160kbps which may be a limitation of how they implemented the host mcu on the bluetooth module, but the datasheet for the bt module is confusing (to me). So even if we could get 350kbps that would be only .87 fps raw (a frame every 1.14 sec) or possibly ~13 fps with jpeg compression.

What I have working now is 160kbps at a decent range. That gives me .39fps or a frame every 2.56 sec which is ~5fps with compression. If I put my api on the computer and use a serial adapter I can get 921600 baud (max of prolific usb converter) and see 2fps. 2fps is much better than a frame every 2.56 sec, and it still sucks !

Options ?

I could use the nordic rf2401 chips and implement a dongle as I had in mind before. I know android devices are getting better at using usb converters and such. This would give me decent performance.

Rosberry pi with wifi.

At the point where I have video working and migrated everything to the F4 where all I had left to do was move it to the android phone (and test) to achieve my milestone I got seriously sidetracked by ROS.

I seem to run into this stumbling block again and again, I get so far along in the project that I already have the next generation in mind and its so much better (in my mind) that I’m prone to throwing away large portions of previous design and coding to move to the “next thing” without achieving the almost completed product or milestone. I am going to stop doing that. I see now that by using ROS and open libraries out there my robot could go much futher than I alone could ever take it; however, I only learned that lesson by going this far down the road I took. I’m sure all of the lessons were not learned yet, so with such a small amount of development left to meet my modest milestone I’m going to revert back to my old code and meet my milestone before going off on any tangents.

Tangents

I can rest a little knowing that all of the ideas I had will be written down and I can pickup where I left off.

Basically everything I have done up until now has already been done and much better in ROS by the amazing people at willow garage. You could sum up my most intricate design into a high power micro pumping video out on rosserial to rosjava_core and viewing this in rviz. Then if you gave me 20 years maybe I could have written all of that code. There is no “game” in ROS, it is real robotics but we can create a subscriber or service that does what we want.

So to sum up how I plan to move forward with ROS after meeting my last milestone is this list.

1. Investigate existing hardware that works with rosserial and could use a camera. Looks like ros works on embedded linux or arduino (but not all arduinos yet). Maybe the rasberry Pi is the best option, since it already could do wifi and embedded linux and would have drivers for the camera
2. Embed ROS or Rosserial into the device, depending on which one you choose. If we go the Pi route we can just run ROS or ROSjava right on the Pi itself. Then we basically have a full robotics platform for $35 plus sd card and wifi, we would want to make it headless eventually.
3. Get the device working with a camera, then publish a camera node in ROS
4. Setup motors and subscribe to a joystick publisher. Joystick to motors.
5. Create clients to view/subscribe to the camera node and publish the joystick node. Maybe we could use rosjava or some other software available and simply point it at the rosmaster? Either way we would want the already predefined and usable ROS format, and we can use the currently available ros tools to test and debug all kinds of stuff.
6. Create easy to use API for beginners so they can easily use our robot and develop their own applications. I know ROS has a learning curve, maybe we could wrap it and make it simpler.

So, I’m going back to meeting my milestone even if it sets me back from ros for a few more weeks. I want to make sure I’ve learned all the lessons of my current path and not just 85% of them. But then I’m going full speed with ROS and more than likely the rasberry pi, which is where I should have started to begin with imho.

Ros/Pi Cost:
Rasberry Pi $35
Wifi WL-700N-RXS $11 (omg you kidding me? 150Mbps!)
Camera board $25 (rPi official camera, expected to be released soon)
Body $30?

All of that adds up to  $101, my goal was always below $100 and that is really close. With the Pi there is the possibility of having multiple cameras and there should be a huge reduction in the amount of development. The bandwidth is almost 200x, the speed is about 10x but the price is only about 2x (for the main board vs stm32 f4).

Granted if I did spend years developing the stm32f103 into something usable it could possibly cost less than $20 for the whole board but in my opinion thats not worth it.

And if I ever want to I can always go back to the Milestone (video and control through phone) and pick up where I left off.

http://wiki.jigsawrenaissance.org/ROS_on_RaspberryPi

http://www.raspberrypi.org/archives/tag/camera-board

http://pingbin.com/2012/12/setup-wifi-raspberry-pi/

http://www.ros.org/wiki/rosserial

I’ve created my first “Java” tutorial. Still not quite finished I want to add motor and gpio control, plus a little more tinsel. But I thought it might be handy to have this here so I can show it to other people.

Basically if you make this in your java main and call setupbot() the bot will set itself up and start polling, then as soon as an image is ready you will get a callback “onnewimage” where you can display it on the screen.

I’m going to add a joystick and some more output to the java frame before I’m done with this example.

public class SimpleBotTutorial implements IBotListener{

ZBot zBot = null;
InputStream inS;
OutputStream outS;

JFrame frame = new JFrame("Video Feed");
JLabel JLabelImage = new JLabel();

public void logI(String str)
{
      System.out.println(str);
}

private boolean getStreams()
{
try
{
      CommPortIdentifier portIdentifier;
      portIdentifier = CommPortIdentifier.getPortIdentifier("COM3");

      CommPort commPort = portIdentifier.open(this.getClass().getName(),2000);
      SerialPort serialPort = (SerialPort) commPort;
      serialPort.setSerialPortParams(baud,SerialPort.DATABITS_8,SerialPort.STOPBITS_1,SerialPort.PARITY_NONE);

      inS = serialPort.getInputStream();
      outS = serialPort.getOutputStream();
      return true;
}
catch (Exception e)
{
      // TODO Auto-generated catch block
      e.printStackTrace();
      // TODO FAIL EXIT
      return false;
}

}

private void setupGui()
{
      frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
      frame.setMinimumSize(new Dimension(251,150));
      frame.getContentPane().add(JLabelImage, BorderLayout.CENTER);
      frame.pack();
      frame.setVisible(true);
}

public boolean startBot()
{
      // if we cant get the stream no reason continuing
      if(!getStreams())
      {
            return false;
      }

      // setup our zbot and pass it the streams
      zBot = new ZBot( inS, outS );
      // tell it we implement its callbacks for video and events
      zBot.setIbl(this);

      // wait a second for a connection, then check
      try {
            Thread.sleep(5000);
      } catch (InterruptedException e) {
            // TODO Auto-generated catch block
      e.printStackTrace();
      }

      // check to see if its ready or we timed out
      if(zBot.getState() < 2)
      {
            // fail
            zBot.pollEngine.stopSvc();
            // close our streams
            try {
                  inS.close();
                  outS.close();
            } catch (IOException e) {
                  // TODO Auto-generated catch block
                  e.printStackTrace();
            }
      // let caller know we failed
      return false;
      }

      // setup the gui to get our data
      setupGui();

return true;
}

public void printAllBotValues()
{
      this.logI("=================================================");
      for ( EzReadVars rv : EzReadVars.values())
      {
            this.logI("Var: " + rv + " Value: " + zBot.getValue(rv));
      }
}

public void rawToImage(byte[] rawData, int w, int h)
{
      // make new container for the image
      BufferedImage NewImg = new BufferedImage(w, h, BufferedImage.TYPE_USHORT_565_RGB);
      // convert to short for raster
      short[] picAsShort = new short[w*h];
      ByteBuffer.wrap(rawData).order(ByteOrder.BIG_ENDIAN).asShortBuffer().get(picAsShort);
      // put directly in raster of image
      short[] imgData = ((DataBufferUShort)NewImg.getRaster().getDataBuffer()).getData();
      System.arraycopy(picAsShort, 0, imgData, 0, picAsShort.length);
      // set our jlabel to the image
      this.JLabelImage.setIcon(new ImageIcon(NewImg));
      // refresh
      frame.repaint();
}

@Override
public void onNewImage(ZImage image) {
      // turn it into a image and display it
      rawToImage(image.getData(),image.getWidth(),image.getHeight());
}

@Override
public void onNewEvent(int event) {
      // TODO Auto-generated method stub
}
}

I thought of several things that I realize that it will be a long time before I’m able to do, things like lasers, navigation, facial recognition, cloud workload balancing and much more. As of now I’m basically building a easy to use RC video Tank that I’m going to put a game on top of.

For some reason I decided to investigate ROS (Robot Operating Sytem) more and now I’m really staring to see how it works. For this reason I’ve put off active development a little while I investigate ROS and make sure my system can be implemented as a ROS package. I believe that using ROS will remove decades of futile “boiler code” development and give me quick access to all of the things that initially got me interested in robotics.

ROS is amazing, it has really blown my mind to see how they have put all of these things together to make a working system. In the ROS system everything is super loosely coupled. Your “robot” is simply a system of talkers, listeners and thinkers or in ROS terms publishers, subscribers and services. I’ve only got about a day of Research in me but I think I have the basic concept. A sensor will talk and “publish” its information at an interval to a “master”. Another subscriber that could be a motor or whatever can subscribe via the “master” to that “publisher” in this way your eyes, ears and sensors are all separate. As far as I can tell a “service” is something that can process hefty data and respond with an answer, like the logic that glues the pieces together.

So how does this all fit in with ZBot ? Well ROS runs under linux so we cant just cram ros into our stm32, we could abandon our platform and use a $30 chip that would run linux and go from there (this is what I plan on experimenting with with my rasberry pi) but we cannot meet our goals with a system that “runs linux”. So, we need to at some point create a “bridge” between our driver and turn our motors and our camera into ROS topics and publish them to the ROS master. This opens the door up to more sensors and things such as odometers, position sensors etc. But more importantly it opens the door up to being able to use those new things and not have to write navigation code and such.

Right now I think the main development will continue as planned, we still want a rc tank with video, but I’m going to be darn sure I’m not spending any time replacing functionality that exists in ROS. Once I get to a point where everything works as I want it too I will write a driver (bridge) for ROS. It will be similar to the NXT driver for ROS. We simply need to add topics for reading our camera and controlling our motors.

ROS actually takes it a step further with their Android “rosjava_core” and their partnership with google. Using these functions you can take a cell phone and open all its sensors up as a ROS topic. I’m currently working on this also. So if my Zbot is a ROS topic and my phone is also then I can simply place my phone on top of my Zbot and be able to retrieve the gps, accelerometer and a faster video stream from the phone itself. All of this data gets shipped over wifi or 4g to a ros master running on some linux server that shares control of the zbot/phone (with all these added sensors). At this point the zbot becomes like a “phone chariot” and we have a robot that can go anywhere (given 4g/gps). Of course all this could be opened up to other nodes such that if there is an alarm the robots investigate, and a service that ships the data to google for cloud processing. Basically everything I could imagine is possible, and I’m now realizing I dont have to do it all !

So I’ll try not to get too sidetracked into ROS, but I want to be sure my zbot is a good fit now so I’m not kicking myself in the butt later. In order to do that I need to investigate a little into the ROS system and set one up etc.

I should be able to meet my milestone of “video on phone, controllable bot” before embarking on the ROS discovery, and I really should so I think thats what I’ll do.