Image Localization

The code below returns the position and orientation of the image (typically from a camera feed) within the map. The LocalizeImage function returns -1 if localization failed, and an integer map ID if it succeeded.

See Immersal.AR.ARLocalizer in the Samples/Scripts/AR folder as an example of how to get the pixel buffer data and intrinsics for the image in an AR Foundation application. Samples/Scenes/SampleScene utilizes this class and can be used for quickly testing of your mappings.

To localize, use this code

int width;          // width of the image
int height;         // height of the image
Vector4 intrinsics; // camera intrinsics
byte[] pixels;      // image data
Vector3 pos;        // resulting device position
Quaternion rot;     // resulting device orientation
​
int mapId = Immersal.Core.LocalizeImage(out pos, out rot, width, height, ref intrinsics, pixels);

Image Capturing

The code below captures and saves a PNG image to the current mapping session. Typically, capturing several images is required to calculate a continuous point cloud map. The CaptureImage function returns an icvCaptureInfo struct with the number of bytes captured (the PNG file size), and information whether the image was connected to a previous image in the map.

The value of channels can be 1 or 3, resulting in an 8-bit grayscale or a 24-bit RGB PNG, respectively.

See the Immersal.Samples.Mapping.Mapper class and Samples/Scenes/MappingApp as an example.

To capture an image, use this code

byte[] pixels;      // image data (typically from the camera)
int width;          // image width
int height;         // image height
int channels;       // 1 or 3, monochromatic or RGB capture
byte[] capture = new byte[channels * width * height + 1024];    // PNG buffer
​
icvCaptureInfo info = Immersal.Core.CaptureImage(capture, capture.Length, pixels, width, height, channels);
int captureSizeInBytes = info.captureSize;
int wasConnected = info.connected;    // 0 if not, 1 otherwise

Getting the Point Cloud

The currently loaded point cloud can be retrieved for e.g. visualization.

The code below retrieves the currently active point cloud and visualizes it. The GetPointCloud function returns the number of points in the cloud.

To get the point cloud, use this code

Vector3[] points = new Vector3[65536];          // allocate space
int num = Immersal.Core.GetPointCloud(points);  // get the point cloud
m_PointRenderer.CreateCloud(points, num);       // do something with the point cloud data, e.g. visualize

Loading the Map

The code below initializes the plugin with a previously captured and saved map data. See Immersal.AR.ARMap in the Samples/Scripts/AR folder as an example.

When using the on-device localizer, the map data (either embedded, or a .bytes file from the Developer Portal) needs to be loaded into the plugin before localization can take place. The LoadMap function returns an integer map ID.

To load the map, use this code

TextAsset cloudFile;
int mapId = Immersal.Core.LoadMap(cloudFile.bytes);

Unloading the Map

The code below unloads a previously-loaded map and frees up the memory. The FreeMap function returns 1 if succeeded, 0 otherwise.

To free a map from the memory, use this code

int mapId = 13;
int result = Immersal.Core.FreeMap(mapId);

Setting and getting plugin parameters

These functions are reserved for setting and getting internal plugin parameters. Currently, only one parameter, LocalizationMaxPixels, is supported. This will effectively downsample any image data to a maximum resolution, which will speed up processing and use less memory -- but might affect localization results. So, experimenting with the values is advised. Setting the value to 0 means "no limit", and is the default setting.

The SetInteger function returns 1 if succeeded, -1 otherwise. The GetInteger function returns the integer value of the parameter, or -1 if not set.

Use the code below to set and get plugin parameters

int result = Immersal.Core.SetInteger("LocalizationMaxPixels", 1280*720);    // use max HD resolution images for matching
if (result == 1)
{
    int newVal = Immersal.Core.GetInteger("LocalizationMaxPixels");
}

Visual GPS functions

The following functions can be used to convert between WGS84, ECEF, and map "cloud space" coordinates. The names of the functions should be quite self-explanatory, but more documentation will follow later on.

public static int MapToEcefGet(double[] mapToEcef, int handle);
public static int PosMapToEcef(double[] ecef, Vector3 map, double[] mapToEcef);
public static int PosEcefToWgs84(double[] wgs84, double[] ecef);
public static int PosWgs84ToEcef(double[] ecef, double[] wgs84);
public static int PosEcefToMap(out Vector3 map, double[] ecef, double[] mapToEcef);
public static int PosMapToWgs84(double[] wgs84, Vector3 map, double[] mapToEcef);
public static int RotMapToEcef(out Quaternion ecef, Quaternion map, double[] mapToEcef);
public static int RotEcefToMap(out Quaternion map, Quaternion ecef, double[] mapToEcef);

Use the code below to get the Visual GPS position after a successful localization

Vector3 pos = new Vector3();
Quaternion rot = new Quaternion();
...
int mapId = Immersal.Core.LocalizeImage(out pos, out rot, width, height, ref intrinsics, pixels);
​
double[] mapToecef = new double[13];
double[] wgs84 = new double[3];
Immersal.Core.MapToEcefGet(mapToecef, mapId);
int r = Immersal.Core.PosMapToWgs84(wgs84, pos, mapToecef);
double vlatitude = wgs84[0];
double vlongitude = wgs84[1];
double valtitude = wgs84[2];