Camera SetUp

We connected the camera to our Arduino I2C input. We ran a script to determine the I2C address of the camera, which was found to be 0x21. Then, we used this data sheet to figure out which registers to set on our camera. Here is a list of registers we had to modify and their functions:

          COM7: 
            Reset all registers 
            Choose camera resolution
            Choose data output format (e.g. RGB)
            Enable color bar test 
          COM3:
            Enable scaling (to change resolution)
            CLKRC
            Use external clock as internal clock
          COM15:
            Choose RGB format (e.g. 565, 555, 444)
          RGB444:
            Enable RGB444 output format
          COM17:
            Enable color bar test 
          COM9:
            Set AGC to 2x and freeze (reduces noise)

Because our onboard memory is less than 300KB and each pixel is stored as 1 byte, we figured that the camera resolution should be set to QCIF (176x144 pixels per frame). Our VGA accepts RGB 332, so we chose RGB as our output format. After trying different combinations of RGB output formats and downsampling, our implementation with RGB444 outputs managed to produce a sharp image that can distinguish red and blue from other colors.

Arduino/FPGA Communication

We wanted to ensure good communication between the FPGA and Arduino while keeping our protocol simple. We originally planned on creating a serial communication protocol, which would result in only having 2 wires between the Arduino and FPGA (A data line and a CLK line). However, upon further inspection, we decided that it would be much easier to use parallel communication with three wires. Each wire represented a bit we were trying to communicate from the FPGA to the Arduino. We only ever need to communicate shape and color to the Arduino. We need two bits for shape because a treasure could be 1 of 3 shapes. We need 1 bit for color because a treasure can only be 1 of 2 colors. This made our communication incredibly simple. The FPGA would just set the wires to be high or low depending on what the camera sees.

FPGA Camera Read-In

In this part of the lab, we chose to use RGB 444 to read in bits bit the OC760, as this gave us more accurate image processing. When set to RGB 444, the OV760 sends out the first 8 bits as XXXXBBBB and the next as RRRRGGGG - contrary to what the OV760 data sheet says. We originally were going to straight downsample the highest bits of each color to fit it to the 8-bit 332 format; however we found it would be more efficient to store all 4 bits of each color which would then be fed to the image processor for more precise color detection. Since we could not use variables in multiple always blocks driven by different conditions, we ended up storing the changes themselves in variables (e.g. VSYNC and lastSYNC) and assigned values based on that. The code snippet below shows under which conditions different variables were updated.

Image Processing

Our image analysis program is tasked with identifying whether or not there is a target on screen, if it is red or blue, and if it is a rhombus, square, or triangle. For this lab we focused on detecting if there was a target, and if so, what color it was. In order for this to work, our camera had to be able to differentiate red and blue colors. Through our FPGA camera read in program above, the camera outputted images where red appeared light red and blue appeared black. Though this was difficult for the human eye, it properly differentiated colors and worked for our image analysis. Our original output images looked like that below.

In order to identify targets we first had to identify which pixels belonged to a target and which did not. Since the blue pixels appeared as black, we set every bit with the Pixel Color 00000000 (black) to 00000011 (blue) in the camera stage and every other bit to 00000000 (black). This changes the output image to one that only shows the blue target in blue and nothing else. We tested this and received the image below. Next we had to set a threshold to find the red target pixels. These pixels had some red in them, Pixel [7:5] > 001, and less green or blue, Pixel [4:2] < 100. Next we set any bit within this threshold to be outputted as bright red (11100000) and all other bits (other than the blue) to be outputted as black. By performing the first step of image analysis in the camera module, debugging the system became easier because we could see what the image analysis was seeing on-screen.

Next we had to detect if a target was on screen. We did this by counting the color pixels in every frame on the V_Sync rising edge. If the number of blue or red pixels were above an empirically measured threshold, there was a blue or red target on screen. If not, there was no target. After tuning the system we were successfully able to detect Red, Blue or no target using the code below.

For the final step we outputted a 3 bit signal to the Arduino through three wires to report the results of the image processor. The lowest bit was the color bit with 1 corresponding to blue and 0 corresponding to red. The second bit was the target bit with zero corresponding to no target and 1 corresponding to target. The highest bit was unused for this current setup but would be implemented once we began to identify shapes as well as color.

The final working version is shown below.