Core Plugin API

Image Localization
The code below returns the position and orientation of the image (typically from a camera feed) within the map. The LocalizeImage function returns  -1 if localization failed, and an integer map ID if it succeeded.
See Immersal.AR.ARLocalizer in the Samples/Scripts/AR folder as an example of how to get the pixel buffer data and intrinsics for the image in an AR Foundation application. Samples/Scenes/SampleScene utilizes this class and can be used for quickly testing of your mappings.
 To localize, use this code
1
int width;          // width of the image
2
int height;         // height of the image
3
Vector4 intrinsics; // camera intrinsics
4
byte[] pixels;      // image data
5
Vector3 pos;        // resulting device position
6
Quaternion rot;     // resulting device orientation
7
​
8
int mapId = Immersal.Core.LocalizeImage(out pos, out rot, width, height, ref intrinsics, pixels);
Copied!
Image Capturing
The code below captures and saves a PNG image to the current mapping session. Typically, capturing several images is required to calculate a continuous point cloud map. The CaptureImage function returns an icvCaptureInfo struct with the number of bytes captured (the PNG file size), and information whether the image was connected to a previous image in the map.
The value of channels can be 1 or 3, resulting in an 8-bit grayscale or a 24-bit RGB PNG, respectively.
See the Immersal.Samples.Mapping.Mapper class and Samples/Scenes/MappingApp as an example.
 To capture an image, use this code
1
byte[] pixels;      // image data (typically from the camera)
2
int width;          // image width
3
int height;         // image height
4
int channels;       // 1 or 3, monochromatic or RGB capture
5
byte[] capture = new byte[channels * width * height + 1024];    // PNG buffer
6
​
7
icvCaptureInfo info = Immersal.Core.CaptureImage(capture, capture.Length, pixels, width, height, channels);
8
int captureSizeInBytes = info.captureSize;
9
int wasConnected = info.connected;    // 0 if not, 1 otherwise
Copied!
Getting the Point Cloud
The currently loaded point cloud can be retrieved for e.g. visualization.
The code below retrieves the currently active point cloud and visualizes it. The GetPointCloud function returns the number of points in the cloud.
 To get the point cloud, use this code

1
Vector3[] points = new Vector3[65536];          // allocate space
2
int num = Immersal.Core.GetPointCloud(points);  // get the point cloud
3
m_PointRenderer.CreateCloud(points, num);       // do something with the point cloud data, e.g. visualize
Copied!
Loading the Map
The code below initializes the plugin with a previously captured and saved map data. See Immersal.AR.ARMap in the Samples/Scripts/AR folder as an example.
When using the on-device localizer, the map data (either embedded, or a .bytes file from the Developer Portal) needs to be loaded into the plugin before localization can take place. The LoadMap function returns an integer map ID.
To load the map, use this code

1
TextAsset cloudFile;
2
int mapId = Immersal.Core.LoadMap(cloudFile.bytes);
Copied!
Unloading the Map
The code below unloads a previously-loaded map and frees up the memory. The FreeMap function returns 1 if succeeded, 0 otherwise.
To free a map from the memory, use this code
1
int mapId = 13;
2
int result = Immersal.Core.FreeMap(mapId);
Copied!
Setting and getting plugin parameters
These functions are reserved for setting and getting internal plugin parameters. Currently, only one parameter, LocalizationMaxPixels, is supported. This will effectively downsample any image data to a maximum resolution, which will speed up processing and use less memory -- but might affect localization results. So, experimenting with the values is advised. Setting the value to 0 means "no limit", and is the default setting.
The SetInteger function returns 1 if succeeded, -1 otherwise. The GetInteger function returns the integer value of the parameter, or -1 if not set.
Use the code below to set and get plugin parameters
1
int result = Immersal.Core.SetInteger("LocalizationMaxPixels", 1280*720);    // use max HD resolution images for matching
2
if (result == 1)
3
{
4
    int newVal = Immersal.Core.GetInteger("LocalizationMaxPixels");
5
}
Copied!
Visual GPS functions
The following functions can be used to convert between WGS84, ECEF, and map "cloud space" coordinates. The names of the functions should be quite self-explanatory, but more documentation will follow later on.
1
public static int MapToEcefGet(double[] mapToEcef, int handle);
2
public static int PosMapToEcef(double[] ecef, Vector3 map, double[] mapToEcef);
3
public static int PosEcefToWgs84(double[] wgs84, double[] ecef);
4
public static int PosWgs84ToEcef(double[] ecef, double[] wgs84);
5
public static int PosEcefToMap(out Vector3 map, double[] ecef, double[] mapToEcef);
6
public static int PosMapToWgs84(double[] wgs84, Vector3 map, double[] mapToEcef);
7
public static int RotMapToEcef(out Quaternion ecef, Quaternion map, double[] mapToEcef);
8
public static int RotEcefToMap(out Quaternion map, Quaternion ecef, double[] mapToEcef);
Copied!
Use the code below to get the Visual GPS position after a successful localization
1
Vector3 pos = new Vector3();
2
Quaternion rot = new Quaternion();
3
...
4
int mapId = Immersal.Core.LocalizeImage(out pos, out rot, width, height, ref intrinsics, pixels);
5
​
6
double[] mapToecef = new double[13];
7
double[] wgs84 = new double[3];
8
Immersal.Core.MapToEcefGet(mapToecef, mapId);
9
int r = Immersal.Core.PosMapToWgs84(wgs84, pos, mapToecef);
10
double vlatitude = wgs84[0];
11
double vlongitude = wgs84[1];
12
double valtitude = wgs84[2];
Copied!

iOS - Previous

Swift

Next - API Documentation

REST API

Last modified 1mo ago

Copy link

Contents

Image Localization

Image Capturing

Getting the Point Cloud

Loading the Map

Unloading the Map

Setting and getting plugin parameters

Visual GPS functions