Module: camera¶

Executive Summary¶

The goal of the camera module is to simulate a camera in the Basilisk codebase. Although images are provided by the visualization, they are renders of the Unity engine and are not necessarily representative of a camera. The module reads in an image from a file, or in the simulation as a pointer to image data, then corrupts it according to input parameters.

Module Assumptions and Limitations¶

The practical limitation of this module is that it decodes and re-encodes the images that are corrupted. Outside of this design choice, the limitations are limited to the corruption methods used to replicate real camera physics. A Gaussian Dark Current might not always be a good model to represent such a phenomenon.

Message Connection Descriptions¶

The following table lists all the module input and output messages. The module msg variable name is set by the user from python. The msg type contains a link to the message structure definition, while the description provides information on what this message is used for

Module I/O Messages¶
Msg Variable Name	Msg Type	Description
imageInMsgName	cameraImageMsg	Input image message This image is theoretically uncorrupted though this module can also add additional errors.
cameraOutMsgName	cameraConfigMsg	Camera parameters message.
imageOutMsgName	cameraImageMsg	Output image with corruptions.

Detailed Module Description¶

Overview of Corruption Sequence¶

This modules pulls heavily from the OpenCV library to perform a series of image corruptions. The following methods are implemented and executed in the sequence shown:

Gaussian noise on the image
Blurring
Dark Current
HSV adjustments
BGR adjustments
Dead and stuck pixels
Random Cosmic rays

Doxygen documentation for OpenCV can be found here.

Overview of Corruptions¶

Gaussian noise is designed to simulate camera sensor noise. This is done by using the addWeighted OpenCV method and scaling the noise according to the input parameter. The noise is zero-mean with standard deviation equal to twice the gaussian noise parameter. The image is then thresholded at the 6-σ value of the noise parameter in order to keep the background dark.
Blurring is implemented using the standard OpenCV blur function with size specified by the blur parameter. This type of blur is referred to as a box blur. The only requirement is that the blur parameter be an odd number.
Dark current is due to thermal properties of the CCD or CMOS sensor in use: as electrons are created independently of incoming light, they are captured in the pixel potential wells and appear to be a signal. Dark current noise is the statistical variation of this phenomenon. In Basilisk, dark current noise is added with a Gaussian noise model with zero standard deviation and mean of 15 × D, where D is another input to the module.
The image color can be changed in the HSV color space. The hue value is rotated by a specified amount of radians, while the saturation and value component can be adjusted through a positive or negative percentage values. The hue value rotates such that 360˚ becomes 0˚. The saturation and value components are limited to [0,255].
The image color can be changed in the BGR color space. All color channel changes are specified through an integer percentage value that can be either positive or negative. The resulting integer color value is limited to lie within [0,255].
Dead and stuck pixels are also implemented as a static perturbation. A dead pixel is a pixel on the sensor that always reads black. A stuck pixel is a pixel on the sensor that always reads white. At the initialization of the run, a random number generates arbitrary pixel coordinates and either saturates them dark or light. The corrupted pixels stay the same throughout the simulation and provide an example of a static artifact.
Camera sensor with relatively high energies, they can saturate a line of pixels and corrupt the image. Cosmic rays are modeled in Basilisk by randomly choosing a point on the sensor as well as second point within a maximal distance of the first one. The abundance of cosmic rays on an image depend on the shutter speed amongst other parameters, and the module allows to toggle the frequency and quantity of such events.

To read more about the corruptions and for example pictures see section 5.2 of Dr. Thibaud Teil’s thesis.

Because each successive filter is applied on top of the previous, the order in which they are applied is very important. Currently Basilisk does not support a custom order with out directly modifying the source code. The order is as shown in the following table describing the filter and the parameters to control the filter. This order was determined in part by trying to match a simulated image to a real image of mars and also based on what makes sense. In the following parameters a value of 0 turns this corruption off and is the default value. Any filter that should be supplied must be provided a non-zero filter parameter value.

Order of Corruptions¶
Corruption	Parameters	Notes
Gaussian Noise	`gaussian`	[double] Adds noise with a mean of 0 and standard deviation of 2 * scaling parameter
Blur	`blurParam`	[double] Determines the size of the box blur. Blur size parameter must be odd
Dark Current	`darkCurrent`	[double] Adds noise with mean of 15 * scaling factor and standard deviation of 0
HSV Adjust	`hsv`	[3D vector of doubles] First parameter is given in radians and determines the hue shift. Second two parameters are scaling factors for saturation and value
BGR Adjust	`bgrPercent`	[3D vector of ints] Parameters correspond to scaling factors for blue, green, and red
Salt/Pepper	`saltPepper`	[double] Probability of both stuck and dead pixels is calculated as 0.00002 * scaling parameter
Cosmic Rays	`cosmicRays`	[double] Adds the specified number of cosmic rays

User Guide¶

The test and these few lines show an example setup for the module.

moduleConfig.imageInMsgName = "sample_image"
moduleConfig.cameraOutMsgName = "cameraOut"
moduleConfig.imageOutMsgName = "out_image"
moduleConfig.filename = ""
moduleConfig.saveImages = 0
# If images are to be saved, add the directory to which they
should be saved
#moduleConfig.saveDir = '/'.join(imagePath.split('/')[:-1]) + '/'

#Camera config values
moduleConfig.cameraIsOn = 1
moduleConfig.sigma_CB = [0,0,1]

#Noise Values
moduleConfig.gaussian = 2
moduleConfig.darkCurrent = 1
moduleConfig.saltPepper = 2
moduleConfig.cosmicRays = 1
moduleConfig.blurParam = 3
moduleConfig.hsv = [30*macros.D2R, 0, 0]

These scalar double values are written such that 0 provides no corruption of that type and 10 provides very high levels of errors (not bounding though)

class Camera : public SysModel ¶

#include <camera.h>

visual camera class

Public Functions

Camera()¶: The constructor for the Camera module. It also sets some default values at its creation.

~Camera()¶: This is the destructor

void UpdateState(uint64_t CurrentSimNanos)¶

This module reads an OpNav image and extracts circle information from its content using OpenCV’s HoughCircle Transform. It performs a greyscale, a bur, and a threshold on the image to facilitate circle-finding.

Return

void

Parameters

CurrentSimNanos: The clock time at which the function was called (nanoseconds)

void SelfInit()¶

Selfinit performs the first stage of initialization for this module. It’s primary function is to create messages that will be written to.

Return: void

void CrossInit()¶

CrossInit performs the second stage of initialization for this module. It’s primary function is to link the input messages that were created elsewhere.

Return: void

void Reset(uint64_t CurrentSimNanos)¶

This method performs a complete reset of the module. Local module variables that retain time varying states between function calls are reset to their default values.

Return

void

Parameters

CurrentSimNanos: current time (ns)

void HSVAdjust(const cv::Mat, cv::Mat &mDst)¶

Adjusts the HSV values of each pixel. Can be used to shift the hue, saturation, and brightness of an image.

Return

void

Parameters

mSrc: source image
mDst: destination of modified image

void BGRAdjustPercent(const cv::Mat, cv::Mat &mDst)¶

Adjusts the BGR values of each pixel by a percent value. Can be used to simulate a sensor with different sensitivities to B, G, and R.

Return

void

Parameters

mSrc: source image
mDst: destination of modified image

void AddGaussianNoise(const cv::Mat, cv::Mat &mDst, double, double)¶

Adds gaussian noise to an image. Can be used to add color noise and dark current.

Return

void

Parameters

mSrc: source image
mDst: destination of modified image
Mean: mean pixel value
StdDev: standard deviation of pixel value

void AddSaltPepper(const cv::Mat, cv::Mat &mDst, float, float)¶

Adds dead and hot pixels to an image.

Return

void

Parameters

mSrc: source image
mDst: destination of modified image
pa: probability of dead pixels
pb: probability of hot pixels

void AddCosmicRay(const cv::Mat, cv::Mat &mDst, float, double, int)¶

Adds a cosmic ray to an image. The ray is modelled as a single pixel wide white line.

Return

void

Parameters

mSrc: source image
mDst: destination of modified image
probThreshhold: probability of getting a ray each frame
randOffset: if adding multiple rays pass in the number of each to guarantee a random ray
maxSize: max length of cosmic ray

void AddCosmicRayBurst(const cv::Mat, cv::Mat &mDst, double)¶

Adds a user specified number of cosmic rays to an image. Rays are modelled as a single pixel wide white line. The maximum length is hard-coded as 50 pixels.

Return

void

Parameters

mSrc: source image
mDst: destination of modified image
num: number of cosmic rays to be added

void ApplyFilters(cv::Mat, cv::Mat &mDst, double gaussian, double darkCurrent, double saltPepper, double cosmicRays, double blurparam)¶

Applys all of the various pertubations to an image with user specified levels. Each parameter is a double scaling actor. A parameter of 0 will result in the respective perturbation not being applied.

Return

void

Parameters

mSource: source image
mDst: destination of modified image
gaussian: scaling factor for gaussian noise
darkCurrent: scaling factor for dark current
saltPepper: scaling factor for hot and dead pixels
cosmicRays: number of cosmic rays to add
blurparam: size of blur to apply

Public Members

std::string filename¶: Filename for module to read an image directly.

std::string imageInMsgName¶: The name of the ImageFswMsg input message.

std::string imageOutMsgName¶: The name of the CameraImageMsg output message.

std::string cameraOutMsgName¶: The name of the CameraConfigMsg output message.

std::string saveDir¶: The name of the directory to save images.

uint64_t sensorTimeTag¶: [ns] Current time tag for sensor out

int32_t saveImages¶: [-] 1 to save images to file for debugging

char parentName[MAX_MESSAGE_SIZE]¶

[-] Name of the parent body to which the camera should be attached

Camera parameters

int cameraIsOn¶: [-] Is the camera currently taking images

int cameraID¶: [-] Is the camera currently taking images

int resolution[2]¶: [-] Camera resolution, width/height in pixels (pixelWidth/pixelHeight in Unity) in pixels

uint64_t renderRate¶: [ns] Frame time interval at which to capture images in units of nanosecond

double fieldOfView¶: [r] camera y-axis field of view edge-to-edge

double cameraPos_B[3]¶: [m] Camera position in body frame

double sigma_CB[3]¶: [-] MRP defining the orientation of the camera frame relative to the body frame

char skyBox[MAX_MESSAGE_SIZE]¶: [-] name of skyboz in use

double gaussian¶

Gaussian noise level.

Noise paramters

double darkCurrent¶: Dark current intensity.

double saltPepper¶: Stuck and Dark pixels probability.

double cosmicRays¶: Random cosmic rays (number)

double blurParam¶: Blur over image in pixels.

std::vector<double> hsv¶: (double) HSV color correction, H (-pi/pi) hue shift, S and V are percent multipliers

std::vector<int> bgrPercent¶: (int) BGR color correction values as percent

BSKLogger bskLogger¶: BSK Logging

Private Members

uint64_t OutputBufferCount¶: [-] Count on the number of output message buffers

int32_t imageInMsgID¶: ID for the outgoing message.

int32_t imageOutMsgID¶: ID for the outgoing message.

int32_t cameraOutID¶: ID for the outgoing message.

uint64_t CurrentSimNanos¶

void *pointImageOut¶: void pointer for image memory passing