Coming from GeomLoss¶
remblais is a drop-in replacement for GeomLoss 0.3.1. It has the same functions, the same signatures, the same calling conventions and the same output shapes; it computes with kernox instead of PyKeOps. Most code only needs its import changed:
# import geomloss
import remblais as geomloss
loss = geomloss.SamplesLoss("sinkhorn", p=2, blur=0.05)
or, function by function:
# from geomloss import SamplesLoss, ImagesBarycenter, sinkhorn_divergence
# from geomloss import ot
from remblais import SamplesLoss, ImagesBarycenter, sinkhorn_divergence
from remblais import ot
As in GeomLoss, ot is a submodule: import it explicitly.
What is the same¶
SamplesLoss: every loss ("sinkhorn","hausdorff","energy","gaussian","laplacian"), every parameter, the three calling conventions, batches,potentials=True, thebackend="auto"rule, custom costs given as KeOps formula strings, custom kernels, explicit cluster labels, and the messages ofverbose=True. Inputs of invalid shapes raise the same errors with the same messages.backend="tensorized"is pure PyTorch, as in GeomLoss: values and gradients are identical bit for bit, apart from the fixes listed below. Theonlineandmultiscalebackends compute the same algorithm with kernox; their results agree with GeomLoss up to rounding.sinkhorn_divergence(images and volumes) andImagesBarycenter.- The
otmodule:solve,solve_batch,solve_sample,OTResult,OTResultMatrix,OTResultSample,LinearOperator. The test suite ofgeomloss.otruns on remblais and passes.
What you do not need any more¶
- PyKeOps, and its compilation toolchain. kernox compiles its kernels at runtime with NVRTC (NVIDIA), hipRTC (AMD) or the oneAPI compiler (Intel), and caches them on disk.
- Recompilations when ε changes: in remblais, ε and the blur are parameters of the compiled reduction, so the whole ε-scaling loop, every call and every batch reuse one kernel.
Things that work differently¶
- GPU vendors. remblais runs on NVIDIA, AMD and Intel GPUs, wherever kernox does.
- CPU. kernox has no native CPU backend. On CPU tensors, the
onlineandmultiscalebackends use the kernox oracle, a slow PyTorch reference: fine for tests, not for speed.tensorizedis unchanged. - Custom costs as Python functions. With the
onlineandmultiscalebackends, a function given ascost(or askernel) receives kernox formulas, not PyKeOpsLazyTensors; see Concepts. KeOps formula strings such as"Norm2(X-Y)"keep working, with every backend. LazyTensor. Theotmodule usesremblais.lazy.LazyTensor, a small kernox-based replacement for the PyKeOpsLazyTensor. It covers whatgeomloss.otneeds: i and j variables and parameters,+ − × ÷,**2,exp,log,sqrt,abs,sum(-1),(x|y), thesum,logsumexp,min,maxandarg*reductions,@,.Tand NumPy inputs. Python numbers become parameters of the compiled reduction.
Deliberate differences¶
These are the places where remblais does not reproduce GeomLoss 0.3.1 on purpose, usually because GeomLoss has a bug there.
- Gradient of unbalanced transport. GeomLoss defines the correction
of the unbalanced gradient (weight ρ + ε) in the
backwardmethod of annn.Module, which PyTorch never calls. Its effective gradient uses ρ + ε/2 and is off by about ε/(2ρ) (3 % for ρ = 1, ε = 0.25). remblais applies the intended correction with atorch.autograd.Function: at convergence, the gradients agree with finite differences to 1e-9. Values and potentials are unchanged, bit for bit. "hausdorff"without a kernel. GeomLoss routes"hausdorff"to the kernel norms, so the call raisesKeyErrorwithout akernel. remblais computes the documented Hausdorff divergence; see Hausdorff divergence. With akernel, the behavior is the same as in GeomLoss.- Grad mode. GeomLoss forces
torch.set_grad_enabled(True)at the end of its Sinkhorn loop, even when called undertorch.no_grad(). remblais restores the state of the caller. - Potentials of kernel losses with
multiscale. GeomLoss returns them in the order of the sorted clusters; remblais returns them in the order of the input points, as GeomLoss already does for Sinkhorn. Likewise,"energy"ortruncate=Nonewithmultiscalereturn a scalar instead of a tensor of shape(1,). - Unbalanced multiscale. GeomLoss reuses the name
epsby mistake while looking for the coarse-to-fine jump, so its final value uses ρ + ε′/2 with ε′ ≠ blurᵖ. remblais uses ε = blurᵖ. - Zero distances. Like GeomLoss with KeOps, the
onlineandmultiscalebackends compute ‖x − y‖ = 0 exactly, with a zero gradient at x = y; thetensorizedbackend keeps the 1e-8 floor of GeomLoss under the square root. - Grid clustering.
grid_clusterlabels the cells by a dense rank obtained by sorting, in lexicographic order like PyKeOps. It has no D ≤ 3 limit and no collisions of the 32-bit integer encoding of PyKeOps. The centroid of a cluster with a single point is that point, exactly. - Dense fallback of multiscale. When the clusters are so small that the
kept blocks would cost more than 0.75 × N · M pair evaluations (counting
each cluster as whole tiles of 128 rows), the fine level is computed on
all pairs, without truncation; the coarse levels are those of GeomLoss.
No term is dropped, so the result is at least as accurate. This happens
typically in 3D with few points per cluster. Set
REMBLAIS_DENSE_RATIO=infto always truncate, as GeomLoss does. - Hard C-transform with
p=1.C_transformon grids raisesNotImplementedErrorforp=1in GeomLoss (anifthat should be anelif); remblais computes it. - Unimplemented
otskeletons.ot.barycenter,ot.barycenter_sample,ot.solve_gridandot.barycenter_gridhave no working body in GeomLoss 0.3.1 (they fail with aNameError); remblais raises an explicitNotImplementedError.
Extensions¶
- Batched custom costs. With
backend="online", a custom cost works with batched inputs(B, N, D); GeomLoss raisesValueError. fast_math. A newSamplesLossargument, described below.
Fast math¶
PyKeOps compiles its kernels with -use_fast_math by default, so the
reductions of GeomLoss use an approximate exp on NVIDIA GPUs. remblais does
the same by default, and makes it explicit with the fast_math argument of
SamplesLoss:
fast_math=None(the default): approximateexpandlogin the kernox reductions on NVIDIA GPUs in FP32; IEEE arithmetic everywhere else (FP64, FP16, AMD, Intel, CPU).fast_math=False: IEEE arithmetic everywhere. Results are then reproducible bit for bit from one run to the next.fast_math=True: asks kernox for fast math explicitly. It has no effect in FP64 and on the CPU oracle, and is refused by the kernox backends that do not offer it.
The relative error of the approximate exp is at most 2⁻²¹ + |x| · 2⁻²⁴
(the kernox contract). sinkhorn_divergence on images, ImagesBarycenter
and the ot module always use IEEE arithmetic.
import torch
from remblais import SamplesLoss
device = "cuda"
x = torch.rand(1_000, 3, device=device)
y = torch.rand(1_000, 3, device=device)
ieee = SamplesLoss("sinkhorn", blur=0.05, fast_math=False)
print(ieee(x, y))