Obtaining External Exposure RGB Images

Application Overview

This application utilizes the MPSizectorS_SDK library to process and read mpdat format floating-point 3D data from the SizectorS 3D camera's external exposure RGB and generates a color image.

Application Process

• Include OpenCV and SizectorS 3D camera-related headers and libraries;

• Call the MPSizectorS_Utils::Load() function to load mpdat data (data type is floating-point 3D);

• Use MPSizectorS_Utils::ConvertToFloat3DFrame() to convert the data into MPSizectorS_DataFrameFloat3DStruct type, facilitating data access;

• Obtain RGB channel grayscale values for each pixel and store them in the rgbImg matrix;

• Use OpenCV's imshow() function to display the RGB color image and the imwrite() function to store the RGB color image;

• Use the waitKey() function to wait for a key press and exit.

Code Invocation

The application mainly consists of the following functions:

Loading Data

This step loads the mpdat-float formatted data file and reads its data, converting it into an MPSizectorS_DataFrameUndefinedStruct type.

MPSizectorS_DataFrameUndefinedStruct data;
bool dev = MPSizectorS_Utils::Load(&data, "../../TestData/normalBox_expo1-result-mode1-rgb.mpdat");

Here, &data is a pointer variable to MPSizectorS_DataFrameUndefinedStruct type.

Format Conversion

This step converts the MPSizectorS_DataFrameUndefinedStruct type data into MPSizectorS_DataFrameFloat3DStruct type data, facilitating subsequent operations to obtain image RGB channel data.

MPSizectorS_DataFrameFloat3DStruct float3DData = MPSizectorS_Utils::ConvertToFloat3DFrame(data);

Obtaining RGB Channel Data

// Create a matrix the same size as the original image, with 3 channels
cv::Mat rgbImg(yPixResolution, xPixResolution, CV_8UC3);

for (int i = 0; i < yPixResolution; i++)
{
	for (int j = 0; j < xPixResolution; j++)
	{
		// Determine if external exposure was enabled during data capture
		bool hasAuxiliaryWhiteImage = false;
		if ((float3DData.FrameInfo.ExternalExposureSettings.ExternalExposureEnable & 0x1) != 0)
		{
			// External white light source exposure was enabled
			hasAuxiliaryWhiteImage = true;
		}

		bool hasAuxiliaryRGBImage = false;
		if ((float3DData.FrameInfo.ExternalExposureSettings.ExternalExposureEnable & 0x2) != 0)
		{
			// External RGB light source exposure was enabled
			hasAuxiliaryRGBImage = true;
		}

		// Data index
		int index = j + i * float3DData.FrameInfo.DataInfo.XPixResolution;

		int dataSize = float3DData.FrameInfo.DataInfo.XPixResolution * float3DData.FrameInfo.DataInfo.YPixResolution;

		// 3D point data
		MPSizectorS_DataPointFloat3DStruct float3DStruct = float3DData.Data[index];

		if (hasAuxiliaryRGBImage)
		{
			cv::Vec3b color_value;

			// The order of external exposure data in memory:
			// dataSize * byte size of W channel data (if available)
			// 3 * dataSize * byte size of RGB channel data (if available)
			// Note: RGB channel data order is dataSize * byte of R channel data, dataSize * byte of G channel data, dataSize * byte of B channel data

			// R channel data
			color_value[2] = float3DData.AuxiliaryRGB[index];
			// G channel data
			color_value[1] = float3DData.AuxiliaryRGB[index + dataSize];
			// B channel data
			color_value[0] = float3DData.AuxiliaryRGB[index + dataSize * 2];

			rgbImg.at<cv::Vec3b>(i, j) = color_value;
		}
	}
}

Storing RGB Color Images

cv::imwrite("rgbImg.png", rgbImg);

Displaying RGB Color Images

cv::namedWindow("rgbImg", cv::WINDOW_NORMAL);
imshow("rgbImg", rgbImg);
cv::resizeWindow("rgbImg", 800, 600);
cv::waitKey();

Here, namedWindow() creates a window with a specified name, and the imshow() function displays the RGB color image in this window.

Output image as follows:

Conclusion

This application demonstrates how to convert external exposure RGB floating-point 3D data from the SizectorS 3D camera into a color image and both display and store the color image.