mypapers/eyecam-fifo/index.html

   1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"
   2            "http://www.w3.org/TR/REC-html40/loose.dtd">
   3 <HTML>
   4 <META NAME="GENERATOR" CONTENT="TtH 2.67">
   5
   6
   7
   8 <title> Speeding up a Digital Camera</title>
   9
  10 <H1 align="center">Speeding up a Digital Camera </H1>
  11
  12 <H3 align=center>2000-05-01 </H3>
  13
  14 <H3 align=center>Petter Reinholdtsen, Stephen T. Humble </H3>
  15
  16
  17 <p>
  18
  19 <H2> Abstract</H2>
  20   To get real time performance when connecting the EyeCam, a VV6300
  21   digital camera to the parallel port of the Eyebot with a reasonable
  22   slow MC68332 CPU, we added flow control by designing a first in
  23   first out buffer to put between the camera and the CPU.
  24 <p>
  25
  26 <p>
  27         <H2><A NAME="tth_sEc1">
  28 1</A>&nbsp;&nbsp;Introduction</H2>
  29
  30 <p>
  31 The Eyebot is a Motorola 68332 based robot controller with on-board
  32 camera support and various input and output connections available.
  33 Initially it used grey scale and color QuickCams.  When Connectix
  34 introduced QuickCam VC without releasing the protocol specification
  35 and hence made it impossible to write drivers, an alternative was
  36 needed.
  37
  38 <p>
  39 The solution was designing the EyeCam, specially made for the Eyebot
  40 and giving much better image quality.  The problem was that the frame
  41 rate dropped to 3.7 frames per second.  The reason for this is that
  42 the EyeCam lacks flow control, and the CPU must synchronize with the
  43 camera to read the data.  This results in approximately 20.000
  44 interrupts for every frame, one for every byte transfered from the
  45 camera.  The EyeCam is capable of running at 60 frames per second, but
  46 this speed was to fast for the CPU.  Processing one interrupt takes to
  47 long, and the data presented by the camera is no longer there when the
  48 CPU tries to read.
  49
  50 <p>
  51 Several solutions to this problem was suggested, among these where
  52 connecting the camera directly to the memory bus, dual ported memory
  53 and adding flow control using a digital  fifo.  This paper
  54 describes the last solution.
  55
  56 <p>
  57         <H2><A NAME="tth_sEc2">
  58 2</A>&nbsp;&nbsp;The camera</H2>
  59
  60 <p>
  61 The EyeCam is made from the VISION VV6300 CMOS Image sensor
  62 [<a href="#vlsi:vv6300" name="CITEvlsi:vv6300">VV6300</a>] and a small support circuit to connect it to the
  63 parallel port (parport).  The camera sensor is no longer produced and
  64 will be replaced with VV6301.  The two sensors are supposed to be
  65 compatible.  As we don't have a VV6301 sensor yet, all our tests are
  66 done with the older VV6300.
  67
  68 <p>
  69 Configuration is done using serial communication on pin  scl and
  70  sda.  Frame start is signaled on the falling edge of pin
  71   fst and every byte is signaled ready on the falling edge of pin
  72  qck.  The frequency of  qck is specified using a clock
  73 divider, and the camera is capable of up to 60 frames per second.
  74
  75 <p>
  76 The camera frame rate is controlled using a clock divisor given as
  77 2<sup>n</sup>.  This clock divisor sets the pixel frequency as shown in table
  78 <A href="#table:clock-divisor">1</A>.  With our camera setup, the  qck
  79 signal will qualify data on the falling edge with this frequency.  If
  80 we assume equal spacing between rising and falling edge, the period
  81 t between falling and rising edge on  qck can be found from the
  82 frequency f<sub>\sc qck</sub> in Hz: t = [1/(2f<sub>\sc qck</sub>)].  We verified the assumption by checking
  83 the  qck signal with an oscilloscope.
  84
  85 <p>
  86 The number of CPU cycles c available to read out the data before
  87 they disappear is then easily calculated using this period t and the
  88 CPU frequency f<sub>\sc cpu</sub> in Hz: c = t &times; f<sub>\sc cpu</sub>
  89
  90 <p>
  91
  92 <p><A NAME="tth_tAb1">
  93 </A> <center> <center>Table 1: Camera timings with 14.318 MHz crystal and 8 bit mode</center>
  94 <A NAME="table:clock-divisor">
  95 </A>
  96
  97 <TaBle>
  98 <tr><td align="right">Clock Divisor           </td><td align="right">2<sup>0</sup> </td><td align="right">2<sup>1</sup> </td><td align="right">2<sup>2</sup> </td><td align="right">2<sup>3</sup> </td></tr>
  99 <tr><td align="right">Data rate (kHz)         </td><td align="right">895 </td><td align="right">448 </td><td align="right">224 </td><td align="right">112 </td></tr>
 100 <tr><td align="right">Frame rate (fps)        </td><td align="right">29.99 </td><td align="right">15.01 </td><td align="right">7.5 </td><td align="right">3.75 </td></tr>
 101 <tr><td align="right">Data present t (ns)   </td><td align="right">559 </td><td align="right">1120 </td><td align="right">2230 </td><td align="right">4460 </td></tr>
 102 <tr><td align="right">Cycles data present c </td><td align="right">19 </td><td align="right">39 </td><td align="right">78 </td><td align="right">156 </td></tr>
 103 <tr><td align="right">Cycles per pixel        </td><td align="right">230 </td><td align="right">459 </td><td align="right">918 </td><td align="right">1861
 104 </td></tr></TaBle>
 105
 106 <p>
 107 </center><p>Table <A href="#table:clock-divisor">1</A> show the results for the possible
 108 clock divisors, assuming CPU running at 35 MHz.  When we know that the
 109 CPU32 platform uses up to 92 cycles to enter and 24 cycles to exit the
 110 interrupt handler [<a href="#mc68300:cpu32" name="CITEmc68300:cpu32">CPU32</a>], and the original optimized
 111 EyeCam driver [<a href="#mcalpine:eyecam" name="CITEmcalpine:eyecam">EyeCam</a>] uses 78 cycles to process one
 112 byte, it is quite impressive that the original EyeCam driver is
 113 working.  It is also obvious that there is no way to increase the
 114 frame rate above 3.7 with the current driver design.
 115
 116 <p>
 117
 118 <p><A NAME="tth_fIg1">
 119 </A>   <center> <img src="ball_072.png" alt="ball_072.png"><br>   <center>Figure 1: Example image taken with the EyeCam</center>
 120   <A NAME="fig:outview">
 121 </A>
 122 </center><p>
 123 <p>
 124 The camera supports two image formats (160 &times;120 and 164&times;124), different transfer modes and 4 and 8 bit pixel sampling.
 125 Each color pixel is the result of combining four camera pixels, one
 126 red, two green and one blue byte.  One image is thus approximately 20
 127 kB.
 128
 129 <p>
 130  qck and  fst can be configured in different modes to change
 131 which edge qualifies the data, and whether control information should
 132 be qualified as well.  The image byte size (as qualified by  qck)
 133 will change when the mode changes.  We found the image size to be
 134 20336 bytes when only image data is qualified and 22352 bytes when
 135 both control and image data are qualified.
 136
 137 <p>
 138 We run the camera in max resolution and 8 bit pixel sampling.  The
 139 frame rate can be doubled by using 4 bit pixel sampling, but when we
 140 tried to use this mode at 3.7 frames per second, the pixel noise made
 141 the images useless.  We do not know why and have not yet tried using
 142 the 4 bit mode at other frame rates.
 143
 144 <p>
 145 The limited processing speed of the current Eyebot platform is a real
 146 challenge.  Still assuming 35 MHz CPU speed, and image size at 82&times;62 pixels, the number of cycles available to process one pixel
 147 at r frames per second is given by the following formula: n = [(f<sub>\sc cpu</sub>)/82 &times;62 &times;r]. The inspiring result
 148 is available in table <A href="#table:clock-divisor">1</A>.  When we know that
 149 CPU32 instructions uses approximately 10 cycles to complete on the
 150 average, real time image processing with even 1861 cycles seems a bit
 151 hard.
 152
 153 <p>
 154
 155 <p><A NAME="tth_tAb2">
 156 </A> <center> <center>Table 2: Camera to PC parallel port control connections</center>
 157 <A NAME="table:connections">
 158 </A>
 159
 160 <TaBle>
 161 <tr><td align="right">Camera pin  </td><td align="right"> scl(1) </td><td align="right"> sda(14) </td><td align="right"> sin(11)
 162      </td><td align="right"> fst(12) </td><td align="right"> qck(10) </td></tr>
 163 <tr><td align="right">Parport pin </td><td align="right"> strobe(1) </td><td align="right"> slctin(17) </td><td align="right"> init(16)
 164      </td><td align="right"> busy(11) </td><td align="right"> ack(10)
 165 </td></tr></TaBle>
 166
 167 <p>
 168 </center><p>The camera control pins are connected to the parport as described in
 169 table <A href="#table:connections">2</A>.  The parport is set up to generate
 170 interrupts and  qck is connected to  ack to trigger the
 171 interrupt when data is ready.
 172
 173 <p>
 174 While working with the  fifo, we discovered a design flaw in the
 175 EyeCam.  The PC parport interrupt is triggering on the rising edge
 176 (0V<font face=symbol>®</font
 177 >5V) of  ack, while the camera qualifies data on
 178 the falling edge (5V<font face=symbol>®</font
 179 >0V) of  qck.  We are not quite
 180 sure why the camera could work anyway, and intend to do some testing
 181 as soon as we find time.  The easiest way to test this would be to
 182 swap  fst and  qck, and run the camera in a different mode
 183 where  fst is the inverted  qck.  We have not investigated
 184 this any further
 185
 186 <p>
 187         <H2><A NAME="tth_sEc3">
 188 3</A>&nbsp;&nbsp;The  fifo</H2>
 189
 190 <p>
 191 The basic idea of the  fifo buffer is to add flow control to the
 192 data stream coming from the camera.  The camera write data into the
 193  fifo at it's configured pace, and the CPU reads them out again
 194 when it has time, and as fast as it can.
 195
 196 <p>
 197 Digital  fifos are available with different capacities and with
 198 different speed requirements.  We only needed to store 8 bits.  The
 199  fifo access time was around 20-50 ns, with the faster
 200   fifos being a lot more expensive than their slower alternatives.
 201 Our access time requirements, as given in table
 202 <A href="#table:clock-divisor">1</A>, would be in the range 500 to 4500 ns.  We
 203 did not find any 8 bit  fifos, and settled on a 9 &times;1024
 204 bit 55 ns access time  fifo in the IC 7200 family for our initial
 205 testings.  It was cheap and easily available over the counter from
 206 Radio Spares.
 207
 208 <p>
 209
 210 <p><A NAME="tth_fIg2">
 211 </A>   <center>  <img src="fifo-simple2.png" alt="fifo-simple2.png"><br>   <center>Figure 2: Outline of  fifo buffer</center>
 212   <A NAME="fig:fifo-outline">
 213 </A>
 214 </center><p>
 215 <p>
 216 The  fifo has five control lines going in; read, write, reset,
 217 retransmit and expand.  All of these are active low.  When running the
 218  fifo in single mode, we do not use retransmit ( rt) and
 219 expand (XI), and these are connected to 5V and 0V respectively.  The
 220 first documentation we got was a bit confusing on this subject, but
 221 after some testing we found this to work.
 222
 223 <p>
 224 The  fifo needs to be reset before it is used, and one must make
 225 sure the read and write signals are high when reset is taken low.  Not
 226 doing proper reset give unpredictable results and gave us a lot of
 227 problems in the beginning.  The  fifo has tree control lines
 228 going out; empty, half-full and full.  On the falling edge of the
 229 half-full line, the  fifo is half full and we can start reading.
 230 When the full line goes low then the  fifo is throwing away all
 231 the new data.
 232
 233 <p>
 234 The first design decision was how to connect the  fifo.  We
 235 decided to continue using the parport, to make it easier to test the
 236 new buffer.  This would make it possible to plug it in between the
 237 current EyeCam and the parport.  The next question was how to connect
 238 the  fifo to the parport and the camera.
 239
 240 <p>
 241 The camera side was fairly straight forward.  The camera data clock
 242 ( qck) needed to connect to the  fifo write pin.  We needed
 243 to make sure they worked with the same edge, and the documentation
 244 (and the oscilloscope) told us that this was the case.  The data
 245 signals where connected to  fifo data pin 0-7 and the
 246 configuration serial lines where connected directly to the parport as
 247 before.  We did not quite know how to connect the camera frame start
 248 ( fst) signal, and tried various solutions.  More on these later.
 249
 250 <p>
 251 Reading the parport specification [<a href="#startech:st16c552" name="CITEstartech:st16c552">ST16C552</a>], we choose
 252 to connect the  fifo control lines reset ( rs), read (
 253   r), empty ( ef), half-full ( hf) and full ( ff) as
 254 shown on figure <A href="#fig:fifo-outline">2</A>.  We decided to connect camera
 255 frame start ( fst) to D<sub>8</sub> and passing it though the  fifo
 256 to parport  busy much later.
 257
 258 <p>
 259 Our first idea for  fst was to connect it as shown on figure
 260 <A href="#fig:fst-init-reset">3</A>.  This way the  fifo would only be reset
 261 between frames if the parport  init was low.  This would also
 262 make sure the  fifo always started reading from the camera on
 263 frame start.  The only problem with using this approach would add one
 264 extra chip to the buffer board, making the buffer bigger and a bit
 265 more expensive, so the choice was easy when we found a way to do
 266 without this chip.
 267
 268 <p>
 269
 270 <p><A NAME="tth_fIg3">
 271 </A>   <center>  <img src="fst-init-reset.png" alt="fst-init-reset.png"><br>   <center>Figure 3:  fst and  init connected to  rs.</center>
 272   <A NAME="fig:fst-init-reset">
 273 </A>
 274 </center><p>
 275 <p>
 276 We tried connecting the  fst directly to the parport  busy,
 277 but was unable to find the proper frame start using this approach.  We
 278 finally decided to connect the  fst to the extra data pin and
 279 change the camera mode to toggle  qck (and therefore write)
 280 during both control and image data.  We later discovered that checking
 281 the  busy line during read was to slow for our purposes and
 282 choose a software only approach for frame synchronization.
 283
 284 <p>
 285 During our initial testing we used a hand-soldered test board with a
 286 1k  fifo.  The initial testing went OK, but the board was fragile
 287 and we often needed to re-solder broken wires.  Later when we had
 288 refined the design a bit, we ran into large problems getting the
 289   fifo to work properly.  We believed this to come from long cables
 290 and problems with  fifo reset, and where close to giving up when
 291 we decided to test with another  fifo - this time a 2k
 292   fifo.  This time the board started working again, and we concluded
 293 that the original  fifo was fried.
 294
 295 <p>
 296 After testing the control lines and struggling with interrupts for a
 297 while, we concluded that the current design should work and got our
 298 local electronic workshop to make a proper  fifo test board.  The
 299 only hardware design-question left was where to place the connectors.
 300 The obvious two options where discussed; placing it on the camera and
 301 placing it on the parport.  We ended up placing it on the parport to
 302 avoid adding more weight to the camera.  The camera connector is as
 303 close to the center of the robot as possible to make sure the cable
 304 would be as short as possible.
 305
 306 <p>
 307
 308 <p><A NAME="tth_fIg4">
 309 </A>   <center>
 310     <center> <img src="fifo-side.png" alt="fifo-side.png"><br>
 311   </center>
 312     <center> <img src="fifo-back.png" alt="fifo-back.png"><br>
 313   </center>
 314
 315  <center>Figure 4: Camera and  fifo test board</center>
 316 </center><p>
 317 <p>
 318 With a proper test board available, things worked even better.  At
 319 this stage we where not sure which edge would trigger the parport
 320 interrupt so we included a jumper to switch from inverted and
 321 non-inverted half-full flag.  We later found out that the PC parport
 322 trigger on the rising edge of  ack, and ended up connecting the
 323 inverted  hf to  ack.  The response time of the inverter can
 324 be ignored, as the half-full flag is set only around 20 times per
 325 frame.
 326
 327 <p>
 328 We tried to use the camera chip ( ce) enable pin to stop the
 329 camera from transmitting data to the  fifo when the full flag
 330 came up, but this did not work.  When we connected  ff to
 331   ce, only black images would come out of the camera.  We tried
 332 contacting VLSI Vision Ltd (which was acquired by STMicroelectronics
 333 in early 1999), to ask about more details on this sparsely documented
 334 part of the camera.  We got one reply with valuable information, but
 335 was also informed that
 336
 337 <p>
 338
 339 <blockquote>  ``STMicroelectronics operates a policy of offering direct support to
 340   high volume customers only.  Support to low volume customers like
 341   yourselves is generally supplied by local distributors.''
 342 </blockquote>
 343
 344 <p>
 345 and were thus unable to investigate any further.  Our local
 346 distributor could not help us at all.
 347
 348 <p>
 349 The buffer still did not work properly.  After a few seconds the
 350 images would turn black, white or with a dithered pattern.  When
 351 checking in the oscilloscope, it seemed like the data did not make it
 352 past the  fifo.  The control signals worked OK, the half-full
 353 flag would signal the driver as expected, but the data was missing.
 354 Looking at the  autofdxt read pin, we suspected a problem with
 355 the shape of the read signal.  When running at full speed, the signal
 356 would be too short and not very square, see figure
 357 <A href="#fig:bad-shape">5</A>.
 358
 359 <p>
 360
 361 <p><A NAME="tth_fIg5">
 362 </A>   <center>  <img src="bad-read-shape.png" alt="bad-read-shape.png"><br>   <center>Figure 5: The not too square  autofdxt read shape</center>
 363   <A NAME="fig:bad-shape">
 364 </A>
 365 </center><p>
 366 <p>
 367 We tried a few things to make the read signal better, but neither
 368 pull-up resistors nor buffers would make it any better.
 369
 370 <p>
 371 During this process we discovered another hardware bug.  The Eyebot MK
 372 3 has pull-up resistors on the open drain parport UART control lines.
 373 It also has a pull-up resistor on parport data pin 2, as a user
 374 discovered and reported.  While we were checking this bug, we found
 375 that this pull-up was supposed to connect to  slctin, and by
 376 accident also discovered that the UART has internal pull-up resistors
 377 on all the open drain lines.  We removed both the bogus and the
 378 duplicate pull-up resistors.  This did not fix the problem with the
 379 read signal.
 380
 381 <p>
 382 As we knew we where pushing the limit for the parallel port, we
 383 decided to try connecting the read line to one of the Eyebots digital
 384 output lines.  This would give us a proper square read signal and the
 385  fifo would keep working.  The only problem with this approach
 386 was that the digital output lines where already used for controlling
 387 the wheel motors, and keeping both the camera driver and the motor
 388 drivers off each others feet would be to slow.
 389
 390 <p>
 391 Our next try was using one of the Eyebots TPU lines to drive the read
 392 pin.  These proved to be to slow.  We could not get them to read out
 393 from the  fifo any faster than approximately 200 kHz.
 394
 395 <p>
 396 When deciding how much data to store in the  fifo, one needed to
 397 think of a few things.  The most important consideration is how much
 398 time is needed for the CPU to start reading from the  fifo.  This
 399 must be less then the time it takes to fill up the second half the
 400  fifo.  This time depends on the interrupt activation time, the
 401  fifo size S<sub>\sc fifo</sub> and the camera write speed
 402 f<sub>\sc qck</sub> and should be as high as possible.
 403
 404 <p>
 405 The next thing to consider is how much latency the  fifo adds to
 406 the images.  When the complete image has been written to the
 407   fifo, it will only be available in memory after it is read out.
 408 This depends on the interrupt activation time and CPU readout speed,
 409 and should be as low as possible.
 410
 411 <p>
 412 To make sure the camera interrupt does not interfere with the other
 413 system interrupts, one call to the interrupt handler should be as
 414 short as possible, yet each call should do enough work to reduce the
 415 overhead per byte transfered.
 416
 417 <p>
 418 One must make sure the  fifo half-full flag is set when one image
 419 is complete.  When the frame size S<sub>frame</sub> isn't a
 420 multiple of half the  fifo size, the simplest way to fix this is
 421 to read out the image size modulo half the  fifo size when
 422 starting to read one frame.  This will make sure the last half-full
 423 flag is set when the last byte of the image is written to the
 424   fifo.  The number of  fifo readouts and thus the number of
 425 interrupts will be <font face=symbol>é</font
 426 >[(2 S<sub>frame</sub>)/(S<sub>\sc fifo</sub>)] <font face=symbol>ù</font
 427 >.
 428
 429 <p>
 430         <H2><A NAME="tth_sEc4">
 431 4</A>&nbsp;&nbsp;The driver</H2>
 432
 433 <p>
 434 We choose to implement the  fifo driver with the same API as the
 435 current RoBIOS camera functions [<a href="#robios:library" name="CITErobios:library">RoBIOS</a>].  The API
 436 consists of CAMInit() to set up the camera plus CAMGetImage() and
 437 CAMGetColImage() to get a 4 bit grey scale or 24 bit color image from
 438 the camera.
 439
 440 <p>
 441 The camera original driver consists of two separate layers, one layer
 442 is the interrupt handler fetching bytes into one ``raw'' buffer, and
 443 one layer to process the ``raw'' bytes into the expected image format.
 444
 445 <p>
 446 While writing the driver, we ran into a few problems.  The most
 447 annoying was trying to get information out of confusing and incorrect
 448 documentation [<a href="#startech:st16c552" name="CITEstartech:st16c552">ST16C552</a>].  It was hard to find out which
 449 control lines where active low, and we ended up testing every pin.
 450 The  fifo outline summarized our findings, with a line about the
 451 pins that are active low.
 452
 453 <p>
 454 The first prototype driver did not use interrupts, but would only
 455 reset the  fifo and wait for the half-full flag to come up before
 456 reading out as fast as possible.  This first naive C version was able
 457 to read at approximately 250 kHz, and we later managed to optimize the
 458 C code to read at approximately 330 kHz.  This driver would constantly
 459 loose frame synchronization, as the  fifo went full between the
 460 reading periods when the images where processed and displayed.  To
 461 solve this problem we needed to get the interrupt triggering going.
 462
 463 <p>
 464 It was very hard to get the half-full flag to trigger the interrupt.
 465 The first problem was finding the correct edge which triggered the
 466 interrupt.  When this was finally solved by adding an inverter between
 467  hf and  ack, we discovered a new problem.  Even if the
 468 interrupts where configured and enabled, the driver would only
 469 sometimes start as it should.  It was as if the first half-full signal
 470 was ignored, and the driver would block forever as the  fifo
 471 already was full and no new interrupt would be triggered.
 472
 473 <p>
 474 The only solution to this problem was to add timeouts to the code and
 475 stop the camera and reset the  fifo when a timeout occurred.
 476 With this in place, the interrupts would work as expected after only a
 477 few resets.
 478
 479 <p>
 480 When we got the interrupt handling working, we discovered that serial
 481 communication no longer worked.  If we stopped the camera interrupt,
 482 everything worked as normal.  The current RoBIOS 3.1 API does not have
 483 a function to signal that the camera is no longer needed.  We suggest
 484 adding a function CAMRelease() to make it possible to stop the camera
 485 to save battery and stop wasting CPU-cycles processing interrupts when
 486 no one will be using the images.  This would also make it possible to
 487 ``enable'' serial communication.
 488
 489 <p>
 490 To let the camera driver handle any  fifo size, and automatically
 491 detect the presence of a  fifo, we came up with the following
 492 procedure.  We configure and start the camera and let it fill up the
 493  fifo (with a timeout until the full flag is set).  Then we stop
 494 the camera and start reading out the data until the empty flag is set.
 495 The number of bytes read should then be the size of the  fifo.
 496 If this procedure fails, timing out while filling up the  fifo,
 497 or being unable to empty the  fifo, the camera is stopped and we
 498 assume there is no  fifo present.
 499
 500 <p>
 501 With the current API, the image data needs to be stored in a image
 502 buffer before it is copied into the user supplied image buffer.  The
 503 current camera API also requires the image bytes to be shuffled to
 504 match the 24 bit image format.  To save processing time, and avoid
 505 copying data, we suggest changing the API to let the user supply
 506 ``raw'' image buffers which are returned when the image bytes are
 507 fetched from the camera.  Then it will be up to the programmer if he
 508 wants to use the ``raw'' camera data or transform it into the standard
 509 24 bit image format.  We therefore propose the following addition to
 510 the RoBIOS camera API:
 511
 512 <p>
 513
 514 <DL compact>  <dt><b><tt>int CAMRawFrameSize(void)</tt></b></dt>
 515         <dd>
 516
 517 <p>
 518   Get the size of the raw camera frames.  This size would be different
 519   for different camera types.
 520
 521 <p>
 522 <dt><b><tt>int CAMPutBuffer(BYTE *buffer)</tt></b></dt>
 523         <dd>
 524
 525 <p>
 526   Give one memory block to the camera driver.  The memory block must
 527   have at least the size of the returned value from <tt>
 528     CAMRawFrameSize()</tt>.
 529
 530 <p>
 531 <dt><b><tt>int CAMBufferReady(void)</tt></b></dt>
 532         <dd>
 533
 534 <p>
 535   Return true (1) if one complete image is available, and false (0) if
 536   no image is ready.
 537
 538 <p>
 539 <dt><b><tt>BYTE *CAMGetBuffer(void)</tt></b></dt>
 540         <dd>
 541
 542 <p>
 543   Get one camera frame from the camera driver.  Block if no camera
 544   frame is available yet, but there are buffers available to fill.
 545   Return <tt>NULL</tt> if there are no more raw camera frames available
 546   to fill.
 547
 548 <p>
 549 <dt><b><tt>CAMBuffer2ColImage(BYTE *buffer, colimage *cimg)</tt></b></dt>
 550         <dd>
 551
 552 <p>
 553   Convert one raw camera frame to the standard 24 bit image format.
 554
 555 <p>
 556 <dt><b><tt>int CAMRelease(void)</tt></b></dt>
 557         <dd>
 558
 559 <p>
 560   Stop the camera and disable parport interrupt.
 561
 562 <p>
 563 </DL>A user application could then use double buffering and avoid any
 564 unneeded pixel copying with C source like this:
 565
 566 <p>
 567
 568 <pre>
 569   int buffersize = CAMRawFrameSize();
 570   BYTE *buffer1 = malloc(buffersize);
 571   BYTE *buffer2 = malloc(buffersize);
 572   /* Initialize camera and prepare double buffering */
 573   CAMInit(NORMAL);
 574   CAMPutBuffer(buffer1);
 575   CAMPutBuffer(buffer2);
 576   while (1) {
 577     if (CAMBufferReady()) {
 578       BYTE *rawframe = CAMGetBuffer();
 579       /* process raw frame */
 580       CAMPutBuffer(rawframe);
 581     }
 582   }
 583   CAMRelease()
 584   free(buffer1);
 585   free(buffer2);
 586 </pre>
 587
 588 <p>
 589         <H2><A NAME="tth_sEc5">
 590 5</A>&nbsp;&nbsp;Conclusion and future work</H2>
 591
 592 <p>
 593 Adding a simple  fifo between the EyeCam and the Eyebot improved
 594 the frame rate and brought us closer to the fundamental limitations of
 595 the platform.  Adding flow control to the camera made it possible to
 596 fetch images faster then it is possible to intelligently process them
 597 on a 35 MHz CPU.
 598
 599 <p>
 600 It would be possible to increase the frame rate even further by
 601 connecting the  fifo directly to the CPU bus in such a way that
 602 every read from the  fifo automatically strobed the read pin.
 603 This way the readout speed would be limited by the speed of one
 604 instruction, instead of the current approach where the read pin must
 605 be manually toggled.
 606
 607 <p>
 608 One way to do this is to connect the parport chip select ( csp)
 609 and one extra address line to the read pin in such a way that the read
 610 pin goes low every time the mirrored parport data address is read.
 611 This requires memory mirroring to work.  We have not investigated
 612 this, as it requires hardware modifications to the Eyebot controller.
 613 Another approach would be to bypass the parport completely, making
 614 sure the  fifo presents data on the low 8 bits on the data bus.
 615
 616 <p>
 617 It might be possible to speed up  fifo reset by connecting the
 618 inverted  fifo reset line to the camera soft reset (SIN) pin.
 619 This way the camera and the  fifo would be reset using one pin,
 620 and without the need to stop the camera.  This requires that setting
 621 the SIN high also sets  qck high.
 622
 623 <p>
 624 Decreasing the image resolution will give more time to process each
 625 pixel, and there are a few simple approaches.  By changing the camera
 626 mode to use ``slow  qck'', only every second byte would be
 627 written to the  fifo, and the resolution would be reduced to 40&times;60 pixels.  This might buy us some extra processing power.
 628
 629 <p>
 630 We do however believe the only solution would be to upgrade the CPU to
 631 a faster and more powerful one is the only solution to be able to do
 632 real time image processing on the Eyebot.  A few alternatives are
 633 available, and we suggest checking out the StrongARM ST-1100
 634 [<a href="#intel:sa1100" name="CITEintel:sa1100">StrongARM</a>] closely.
 635
 636 <p>
 637 <H2>References</H2>
 638 <DL compact>
 639
 640 <p>
 641 <dt>[<a href="#CITEvlsi:vv6300" name="vlsi:vv6300">VV6300</a>]</dt><dd>
 642   VLSI VISION Limited:
 643   VV6300, Low Resolution Digital CMOS Image Sensor (1998).
 644
 645 <p>
 646 <dt>[<a href="#CITEamd:am7203a" name="amd:am7203a">AM7203A</a>]</dt><dd>
 647   Advanced Micro Devices:
 648   Am7203A, High Density First-In First-Out (FIFO) 2048x9Bit CMOS
 649   Memory (1992).
 650
 651 <p>
 652 <dt>[<a href="#CITEstartech:st16c552" name="startech:st16c552">ST16C552</a>]</dt><dd>
 653   Startech:
 654   ST16C552, Universal Asynchronous Receiver/Transmitter with FIFO and
 655   Parallel Printer Port with Power Down Capability (1996).
 656
 657 <p>
 658 <dt>[<a href="#CITEmcalpine:eyecam" name="mcalpine:eyecam">EyeCam</a>]</dt><dd>
 659   McAlpine, Paul R.:
 660   Image Processing and CMOS Camera Integration with the EyeBot
 661   Platform, Honours Dissertation (1999).
 662
 663 <p>
 664 <dt>[<a href="#CITErobios:library" name="robios:library">RoBIOS</a>]</dt><dd>
 665   T. Bräunl, K. Schmitt, T Lampart:
 666   Eyebot RoBIOS library documentation (1998).
 667   <a href="http://www.ee.uwa.edu.au/~braunl/eyebot/ftp/ROBIOS/library.html"><tt>http://www.ee.uwa.edu.au/~braunl/eyebot/ftp/ROBIOS/library.html</tt></a>
 668
 669 <p>
 670 <dt>[<a href="#CITEmc68300:cpu32" name="mc68300:cpu32">CPU32</a>]</dt><dd>
 671   Motorola:
 672   M683000 Family CPU32 Central Processor Unit Reference Manual (1990).
 673
 674 <p>
 675 <dt>[<a href="#CITEintel:sa1100" name="intel:sa1100">StrongARM</a>]</dt><dd>
 676   Intel:
 677   Intel StrongARM SA-1100 Microprocessor for Portable Applications (1999)
 678
 679 <p>
 680 </DL>
 681 <H2>Acknowledgments and contact information</H2>
 682
 683 <p>
 684 Thanks to Thomas Bräunl, Director of the Mobile Robots Lab and creator
 685 of the EyeBot robot family, for giving us the possibility to play with
 686 the robots in the first place.  Thanks to Klaus Schmitt for all his
 687 help with the parport interrupt handling.  Thanks to Ivan Naubronner
 688 for giving all his help on the electronic design.
 689
 690 <p>
 691 Thanks to Ken Cormack at STMicroelectronics for valuable information
 692 on the VV6300 sensor, despite the company policy to not help the low
 693 volume costumers.
 694
 695 <p>
 696 Petter Reinholdtsen can be reached via email to
 697 <a href="mailto:pere@td.org.uit.no">pere@td.org.uit.no</a>.  Stephen T. Humble can be reached via
 698 email to <a href="mailto:steve@newton.dialix.com.au">steve@newton.dialix.com.au</a>.
 699
 700 <p>
 701 Information on the EyeCam and the Eyebot is available from<br>
 702 <a href="http://www.ee.uwa.edu.au/~braunl/eyebot/"><tt>http://www.ee.uwa.edu.au/~braunl/eyebot/</tt></a>.
 703
 704 <p>
 705 <hr>
 706
 707 <p>
 708 This article was rejected when submitted for RoboCup 200 workshop.
 709 Here is the rejection letter.  I was initially planning to update the
 710 paper based on the comments, but later discovered I would never take
 711 the time required to do it.
 712
 713 <p>
 714
 715 <pre>
 716 From: Gerhard Kraetschmar &lt;gkk@acm.org&#62;
 717 Date: Thu, 22 Jun 2000 23:33:16 +0200
 718 Message-Id: &lt;3952861C.22CE96DD@acm.org&#62;
 719 To: pere@hungry.com
 720 Subject: Notification of Workshop RoboCup-2000 submission
 721
 722 Dear author,
 723
 724 we regret to inform your paper has not been accepted for presentation
 725 at the RoboCup-2000 Workshop.
 726
 727 Because the number of submissions to this workshop has grown
 728 in recent years, we have had to decline many papers that would have been
 729 accepted previously.
 730 We had a large number of high quality submissions and an acceptance
 731 rate of about 30% for oral presentation.
 732
 733 Reviews of your articles are attached below.
 734
 735 We look forward to a very exciting and successful workshop and
 736 sincerely hope that you will be able to attend.
 737
 738 Best regards,
 739         Peter, Tucker, and Gerhard
 740
 741 ==============================================================
 742 RoboCup 2000 Paper Review Form
 743
 744 Code: RK-9
 745 Title: Speeding up a digital camera
 746 Authors:  Reinholdtsen, Humble
 747
 748 COMMENTS FOR AUTHORS
 749 Recommendation:
 750      ___ Accept  ___ Marginal accept ___ Marginal reject      X    Reject
 751
 752 Confidence in your recommendation:
 753     X     High     ___ Medium       ___ Low
 754
 755 Is the paper a good fit to the symposium?
 756      ___ Yes      ___ Somewhat     X  No     ___ Unsure
 757
 758 Is the work sufficiently novel to warrant publication?
 759      ___ Yes      X Somewhat     ___ No     ___ Unsure
 760
 761 Does it provide a reasonable amount of detail for a workshop paper?
 762      ___ Yes      X  Somewhat     ___ No     ___ Unsure
 763
 764 Are the implementation and experimentation sufficient? (if
 765 applicable)  Keep in mind that the paper may describe work in
 766 progress.
 767      ___ Yes      X  Somewhat     ___ No     ___ Unsure
 768
 769 Are the technical sections well-formed and accurate?
 770      ___ Yes      X Somewhat     ___ No     ___ Unsure
 771
 772 Is the presentation well organized?
 773      ___ Yes      ___ Somewhat     X  No     ___ Unsure
 774
 775 Is the literature survey adequate for a conference paper?
 776      ___ Yes      ___ Somewhat     X  No     ___ Unsure
 777
 778 What did you like best about this paper?
 779 It  may  be an  interesting  paper  about  designing digital  cameras,
 780 provided the  paper is cleaned up  by removing all  "fairy tale" style
 781 and narration about partial failures  (which may be good in a tutorial
 782 paper, but not in a scientific one).
 783 However, the  only reference to the  RoboCup is the  picture in fig.1,
 784 whose quality is quite questionable anyway.
 785
 786 When revising this paper, what do you think the author(s) should give
 787 highest priority to?
 788 If the  paper is  accepted, it should  definitely point out  much more
 789 clearly what contribution  this work may provide to  RoboCup teams and
 790 RoboCup-related research.
 791 English should also be revised.
 792
 793 Other comments for authors (elaborate on any negative rankings above):
 794 The  paper is  of very  marginal  interest to  the RoboCup  community,
 795 unless some  specific experimentation is  carried out. But  that would
 796 mean re-writing  almost the whole  paper. In this format,  the RoboCup
 797 workshop audience is not the right target for this paper.
 798
 799 ==============================================================
 800
 801 ==============================================================
 802 RoboCup 2000 Paper Review Form
 803
 804 Please return your reviews by June 5th, 2000 to gkk@acm.org
 805
 806 Code: RK-9
 807 Title: Speeding up a digital camera
 808 Authors:  Reinholdsten, Humble
 809
 810
 811 COMMENTS FOR AUTHORS
 812
 813
 814 Recommendation:
 815      ___ Accept   ___ Marginal accept ___ Marginal reject  x___ Reject
 816
 817 Confidence in your recommendation:
 818      x___ High     ___ Medium       ___ Low
 819
 820 Is the paper a good fit to the symposium?
 821      ___ Yes      ___ Somewhat    x ___ No     ___ Unsure
 822
 823 Is the work sufficiently novel to warrant publication?
 824      ___ Yes      ___ Somewhat     x___ No     ___ Unsure
 825
 826 Does it provide a reasonable amount of detail for a workshop paper?
 827      x___ Yes      ___ Somewhat     ___ No     ___ Unsure
 828
 829 Are the implementation and experimentation sufficient? (if
 830 applicable)  Keep in mind that the paper may describe work in
 831 progress.
 832      ___ Yes      x___ Somewhat     ___ No     ___ Unsure
 833
 834 Are the technical sections well-formed and accurate?
 835      ___ Yes      x___ Somewhat     ___ No     ___ Unsure
 836
 837 Is the presentation well organized?
 838      ___ Yes      x___ Somewhat     ___ No     ___ Unsure
 839
 840 Is the literature survey adequate for a conference paper?
 841      ___ Yes      x___ Somewhat     ___ No     ___ Unsure
 842
 843 What did you like best about this paper?
 844
 845 I'm a hardware hobbyist, so I like the work that was done to interface
 846 a digital camera to a simple microprocessor.  But using a FIFO to
 847 accomodate peripherals with that are asynchronous to the system bus is
 848 a very standard idea.  It is also not appropriate for this workshop.
 849
 850
 851 When revising this paper, what do you think the author(s) should give
 852 highest priority to?
 853
 854
 855 Other comments for authors (elaborate on any negative rankings above):
 856
 857 I like the work, but it would be better suited to a hardware
 858 conference specifically devoted to microprocessor issues.  If you're
 859 looking for a better image processor, you might try DSP chips like the
 860 Analog Devices ADSP2181, which have direct DMA from external devices
 861 into internal memory, and much more processing power.
 862 ==============================================================
 863 </pre>
 864
 865 <p>
 866
 867 <p><hr><small>File translated from
 868 T<sub><font size="-1">E</font></sub>X
 869 by <a href="http://hutchinson.belmont.ma.us/tth/">
 870 T<sub><font size="-1">T</font></sub>H</a>,
 871 version 2.67.<br>On 19 Sep 2000, 17:57.</small>
 872 </HTML>