calibur package#

Submodules#

calibur.resources module

Module contents#

class calibur.GraphicsNDArray(input_array)#

A subclass of numpy.ndarray.

It adds GLSL-like access to components, e.g. a.xyz / a.w is a shorthand for a[..., [0, 1, 2]] / a[..., [3]].

Note that single-component results retains a singleton 1 dimension.

__init__()#

class calibur.BVH(triangles: ndarray, centroids: ndarray | None = None, indices: ndarray | None = None)#

Bases: object

__init__(triangles: ndarray, centroids: ndarray | None = None, indices: ndarray | None = None) → None#

Construct BVH from triangles.

Parameters:

triangles – (N, 3, 3) array of 3 vertices.
centroids – See note below.
indices – See note below.

Centroids and indices should only be passed if it is not the root node, which should not be used other than by the BVH itself in principle.

raycast(rays_o: ndarray, rays_d: ndarray, far=32767.0)#

Cast a batch of rays to the triangles in the BVH.

Parameters:

rays_o – (N, 3) ray origins.
rays_d – (N, 3) ray directions (normalized).

Returns:

a tuple (hit_i, hit_d, hit_u, hit_v).

hit_i – (N) int32, index into given triangles, -1 if missing.
hit_d – (N) float32, ray travel distances.
hit_u – (N) float32, triangle barycentric u.
hit_v – (N) float32, triangle barycentric v.

For barycentric coordinates, P = w v1 + u v2 + v v3.

calibur.CC#: alias of CameraConventions

class calibur.CameraConventions#

Bases: object

Conventions of camera coordinate systems in right, up, forward order. Also available as CC for faster typing.

Blender = ('X', 'Y', '-Z')#

CV = ('X', '-Y', 'Z')#

DirectXLH = ('X', 'Y', 'Z')#

GL = ('X', 'Y', '-Z')#

Godot = ('X', 'Y', '-Z')#

OpenCV = ('X', '-Y', 'Z')#

OpenGL = ('X', 'Y', '-Z')#

ROS = ('-Y', 'Z', 'X')#

UE = ('Y', 'Z', 'X')#

Unity = ('X', 'Y', 'Z')#

class calibur.Environment#

Bases: object

Abstract base class for an shading environment.

Available implementations: NormalCaptureEnvironment, SHEnvironment, SampleEnvironments.

shade(normals: ndarray)#

Parameters:: normals – (N, 3) world-space normals.
Returns:: (N, 3) RGB values in [0, 1] range.

class calibur.NormalCaptureEnvironment#

Bases: Environment

A special ‘environment’ to capture the world-space normals of rendered object.

shade(normals)#

Parameters:: normals – (N, 3) world-space normals.
Returns:: (N, 3) RGB values in [0, 1] range.

class calibur.NumPyWarning(all: Literal['ignore', 'warn', 'raise', 'call', 'print', 'log', None] | None = None, divide: Literal['ignore', 'warn', 'raise', 'call', 'print', 'log', None] | None = None, over: Literal['ignore', 'warn', 'raise', 'call', 'print', 'log', None] | None = None, under: Literal['ignore', 'warn', 'raise', 'call', 'print', 'log', None] | None = None, invalid: Literal['ignore', 'warn', 'raise', 'call', 'print', 'log', None] | None = None)#

Bases: object

__init__(all: Literal['ignore', 'warn', 'raise', 'call', 'print', 'log', None] | None = None, divide: Literal['ignore', 'warn', 'raise', 'call', 'print', 'log', None] | None = None, over: Literal['ignore', 'warn', 'raise', 'call', 'print', 'log', None] | None = None, under: Literal['ignore', 'warn', 'raise', 'call', 'print', 'log', None] | None = None, invalid: Literal['ignore', 'warn', 'raise', 'call', 'print', 'log', None] | None = None) → None#

Context manager for numpy warnings. Possible options are:

'ignore'
'warn'
'raise'
'call'
'print'
'log'

class calibur.SHEnvironment(sh: ndarray, world_convention)#

Bases: Environment

__init__(sh: ndarray, world_convention) → None#

The environment lighting method by Ramamoorthi et al. “An Efficient Representation for Irradiance Environment Maps” (2001). The environment light only depends on surface normal, and is represented by Spherical Harmonics (SH) coefficients.

Parameters:

sh – (9, 3) SH coefficients of the environment map.
world_convention – target world convention for shading.

Two built-in environments can be found at SampleEnvironments.

classmethod from_image(img: ndarray, world_convention)#

Box-filtered integration of equirect environment map into SH coefficients.

Parameters:

img – (H, W, C), preferably W = 2H.
world_convention – target world convention for shading.

Experimental.

shade(normals)#

Parameters:: normals – (N, 3) world-space normals.
Returns:: (N, 3) RGB values in [0, 1] range.

class calibur.SampleEnvironments#

Bases: object

Example environments in the paper “An Efficient Representation for Irradiance Environment Maps” (2001).

static eucalyptus_grove(world_convention)#

static grace_cathedral(world_convention)#

class calibur.SimpleRayTraceRP#

Bases: object

A minimal CPU ray tracing render pipeline for debugging cameras.

render(env: Environment, cam_pose_cv: ndarray | GraphicsNDArray, fx: float, fy: float, cx: float, cy: float, sx: float, sy: float, pixel_per_unit: float = 1.0)#

Renders the geometry using an environment.

Parameters:

env – The Environment to shade the world.
cam_pose_cv – (4, 4) camera pose in CameraConventions.CV conventions.
fx – Focal length on the x (horizontal) axis.
fy – Focal length on the y (vertical) axis.
cx – Principal point x (horizontal) axis. Origin is top-left of frame.
cy – Principal point y (vertical) axis. Origin is top-left of frame.
sx – Sensor size x (horizontal), or width.
sy – Sensor size y (vertical), or height.
pixel_per_unit – If the previous focal lengths and sizes are in physical units like mm, a value can be specified for pixel density per unit. Defaults to 1.0.

Returns:

(H, W, 3) RGB image ranging in [0, 1], where [H W] = round([sy sx] * pixel_per_unit).

set_geometry(tris: ndarray | GraphicsNDArray)#

Set geometry triangles. Assumes CCW winding.

Parameters:: tris – (N, 3, 3) triangles, array of 3 vertices.
Returns:: self after preprocessing the geometry.

class calibur.WorldConventions#

Bases: object

Conventions of world coordinate systems in right, up, forward order.

Currently experimental, as the definition of world forward is tricky. The up axis is in general accurate and not ambiguous.

The current conventions for listing is:

Up is for gravity up.
Forward is: when you look into forward, you see the front of objects. This means that humanoid or animals walk in the backward direction.
Right is: when you look into forward, the right of you.

As a result, when dealing with humanoids in game engines like unity, right/forward can be the inverse direction to your expectations. Unfortunately, this mirror effect exists in real-world and cannot be further unified.

Blender = ('X', 'Z', 'Y')#

DirectXLH = ('X', 'Y', '-Z')#

GL = ('X', 'Y', '-Z')#

GLTF = ('X', 'Y', '-Z')#

Godot = ('X', 'Y', '-Z')#

ROS = ('-Y', 'Z', 'X')#

UE = ('X', 'Z', '-Y')#

Unity = ('X', 'Y', 'Z')#

calibur.backcaster(arr)#: Cast a GraphicsNDArray into numpy.ndarray, but keeps other types unchanged.

calibur.cast_graphics(func: T) → T#: Decorator that casts all arguments into GraphicsNDArray.

calibur.caster(arr)#: Cast a numpy.ndarray into GraphicsNDArray, but keeps other types unchanged.

calibur.compute_tri3d_normals(tris: ndarray | GraphicsNDArray)#

Compute face normals of triangles.

Assumes the CCW order as forward face.

Parameters:: tris – (..., 3, 3) where the last dimension is xyz and the second last is 3 vertices.
Returns:: (..., 3) of normals of each triangle.

calibur.container_catamorphism(data, func)#: Transforms leaf elements in list, dict, tuple, set with func, aka. tree-map. Nested containers are also supported.

calibur.convert_pose(src_pose, src_convention, dst_convention)#

Converts a pose between conventions. Think of a pose as:

A small object has a fixed relative position to the camera. The camera pose transforms the relative position into a world position.

A @ p_c = p_w

The conversion works as:

We have a camera with pose A in system A (say OpenGL). Now we instead want a camera in system B (say OpenCV) to see the same scene. What should be its pose? We want to transform a point with same right, up and forward relative to the camera into the same world point. The answer would be convert_pose(A, CC.GL, CC.CV).

Parameters:

src_pose – (..., 4, 4) pose to convert.
src_convention – Source convention of src_pose. CameraConventions or WorldConventions.
dst_convention – Destination convention to convert into. CameraConventions or WorldConventions.

Returns:

(..., 4, 4) converted pose.

calibur.focal_to_fov(focal_length, size)#: Inverse of fov_to_focal. Angles are in radians.

calibur.fov_to_focal(fov, size)#

Compute fov y and size h to focal y,

Compute fov x and size w to focal x.

Parameters:

fov – Horizontal or vertical FOV angle in radians.
size – Width or height.

Returns:

Focal length in the same unit as size.

calibur.fov_y_to_x(fovy, aspect)#

Compute fov x from y.

Parameters:

aspect – W / H.
fovy – Vertical FOV angle in radians.

Returns:

Horizontal FOV angle in radians.

calibur.get_cam_rays_cv(c2w, fx, fy, cx, cy, h, w)#

Get camera rays (origin, dir) from extrinsics and intrinsics in OpenCV convention.

Returns:

a tuple (rays_o, rays_d).

rays_o – (h * w, 3), the origin of the rays in world coordinates.
rays_d – (h * w, 3), the normalized direction of the rays in world coordinates.

calibur.get_dx_viewport_rays(h, w, z_start)#

Shoot rays from z_start towards z +inf in viewport space.

Viewport xy lies in (0-w, 0-h), where 0/w/h represent borders (instead of center of border pixels) of viewport.

The origin is top-left (vulkan/directx convention).

Returns:: a tuple (rays_o, rays_d), both of shape (h * w, 3).

calibur.get_ruf_basis(convention)#: Get the right/up/forward basis matrix in convention. The matrix is laid out as [r|u|f].

calibur.get_view_ray_directions_cv(h, w, fx, fy, cx, cy, norm=False)#

Parameters:

h – (int) height of camera.
w – (int) width of camera.
intrinsics – [fx, fy, cx, cy] CV intrinsics of camera.
norm – whether the ray directions should be normalized.

Returns:

(h, w, 3) directions of the rays in OpenCV camera coordinate

calibur.gl_ndc_to_dx_ndc(ndc: ndarray | GraphicsNDArray)#

DX NDC differs to GL in that depth lies in [0, 1] instead of [-1, 1].

This is a simple transform compressing the Z dimension of NDC.

calibur.gl_ndc_to_dx_viewport(ndc: ndarray | GraphicsNDArray, w: float, h: float, near=None, far=None)#

GL NDC to DX viewport coordinates.

If near and far are not None, the resulting Z is linear depth. Otherwise Z is kept unchanged.

calibur.gl_ndc_to_gl_viewport(ndc: ndarray | GraphicsNDArray, w: float, h: float, near=None, far=None)#

GL NDC to GL viewport coordinates.

Notice that GL viewport origin is the bottom-left corner.

If near and far are not None, the resulting Z is linear depth. Otherwise Z is kept unchanged.

calibur.gl_viewport_to_dx_viewport(vp: ndarray | GraphicsNDArray, h: float)#: GL viewport to DX/Vulkan viewport coordinates.

They differ in that DX viewport origin is top-left compared to bottom-left in GL.

calibur.homogeneous(coords: ndarray | GraphicsNDArray)#: Concatenate a 1 dimension to the coords to transform it into homogeneous coordinates.

calibur.intrinsic_cv(cx, cy, fx, fy, s=0.0)#

OpenCV intrinsic matrix construction. Input units are usually pixels for CV.

Parameters:

cx – Principal point x (horizontal).
cy – Principal point y (vertical).
fx – Focal length x (horizontal).
fy – Focal length y (vertical).
s – Camera shear.

Returns:

(4, 4) GL projection matrix

calibur.linear_depth_gl(ndc_z, near, far)#: GL NDC Z to linear depth formula.

calibur.magnitude(vecs: ndarray | GraphicsNDArray)#

Compute the magnitudes or lengths of vecs.

Parameters:: vecs – (..., D).
Returns:: (..., 1).

calibur.normalized(vecs: ndarray | GraphicsNDArray)#

Safely normalize a batch of vecs.

Parameters:: vecs – (..., D).
Returns:: (..., D), the same shape as input.

Decide whether points lie in 2D triangles. Arrays broadcast.

Parameters:

pt – (..., 2) points to decide.
v1 – (..., 2) first vertex in triangles.
v2 – (..., 2) second vertex in triangles.
v3 – (..., 2) third vertex in triangles.

Returns:

(..., 1) boolean result of points lie in triangles.

calibur.projection_gl_persp(w, h, cx, cy, fx, fy, near, far, s=0.0)#

Gets the GL projection P matrix from camera intrinsics. The function is unit-insensitive – the input c/f/w/h values can be in any unit as long as they are the same (pixel/mm/inch/etc).

camera_view_space_pts @ P.T -> gl_clip_space_pts.

Applying .xyz / .w gives the GL NDC coordinates:

.x: left to right => -1 to 1

.y: bottom to up => -1 to 1

.z: near to far => -1 to 1

Parameters:

w – Width.
h – Height.
cx – Principal point x (horizontal).
cy – Principal point y (vertical).
fx – Focal length x (horizontal).
fy – Focal length y (vertical).
near – Camera near clipping plane distance.
far – Camera far clipping plane distance.
s – Camera shear.

Returns:

(4, 4) GL projection matrix

calibur.sample2d(im: ndarray | GraphicsNDArray, xy: ndarray | GraphicsNDArray) → ndarray | GraphicsNDArray#

Bilinear sampling with UV coordinate in Blender convention.

The origin (0, 0) is the bottom-left corner of the image as in most UV conventions.

Parameters:

im – (H, W, ?) image.
xy – (..., 2), should lie in [0, 1] mostly (out-of-bounds values are clamped).

Returns:

(..., ?) sampled points.

calibur.supercat(tensors: Sequence[ndarray], dim: int = 0)#

Similar to numpy.concatenate, but supports broadcasting. For example:

[M, 32], [N, 1, 64] -- supercat 2 --> [N, M, 96]

calibur.transform_point(xyz: ndarray | GraphicsNDArray, matrix: ndarray | GraphicsNDArray) → ndarray | GraphicsNDArray#

Transforms points with transformation matrices. The function is not limited to 3D coords.

The output is normalized to w=1 in homogeneous coordinates.

Parameters:

xyz – (..., D)
matrix – (..., D + 1, D + 1)

Returns:

(..., D)

calibur.transform_vector(xyz: ndarray | GraphicsNDArray, matrix: ndarray | GraphicsNDArray) → ndarray | GraphicsNDArray#

Transforms directions with transformation matrices. The function is not limited to 3D coords.

Parameters:

xyz – (..., D)
matrix – (..., D + 1, D + 1)

Returns:

(..., D)

calibur.treemap_backcaster(what)#: Cast any GraphicsNDArray in (nested) lists/dicts/tuples/sets into numpy.ndarray, and keeps other types unchanged.

calibur.treemap_cast_graphics(func: T) → T#: Decorator that casts all arguments of type numpy.ndarray into GraphicsNDArray. Also casts arrays in nested lists, dicts or tuples.

calibur.unbind(arr: ndarray, axis, keepdims=False)#: Unbinds an NDArray into a list of arrays along an axis.