Nishanth Sundaran

Its design starts from frax (Stanford ASL): the dense, loop-free formulation of the dynamics over a DOF ancestor mask. rigidax fills in what frax lacks: it loads MJCF and URDF through MuJoCo, supports ball joints and several joints per body, makes gravity a model parameter, and keeps masses, inertias and offsets as pytree leaves, so jax.grad and vmap work over the physical parameters without recompiling. Every quantity is tested against MuJoCo on random trees and the G1, A2 and Panda, plus Pinocchio. On top of it sits a batched IK retargeter for LAFAN1. Both libraries ran frax’s own LAFAN1 settings on the same laptop, on 8 clips (53,621 frames, 30 minutes of motion), scored with the same metrics: with self-collision rigidax runs at 22,836 frames/s against 18,327, the toe slides 0.9 cm/s median against 4.6 while the human toe is planted, frames with a joint jump over 20° fall from 1.9% to 0.1%, and the lag goes from 0.16 frames to zero. frax is still slightly ahead on forearm twist and on leg motion kept. The G1 wears Dex3 hands, and since LAFAN1 records no fingers the grip is inferred from context and written after the body solve, so the hands cost almost nothing (18.8k frames/s). An operational space controller on the Panda, with all its dynamics from rigidax, tracks a figure-eight at 0.9 mm mean error in about 54 µs per step.

Built with JAX, MuJoCo, Pinocchio, NumPy, pytest, GitHub Actions, PyPI trusted publishing and Python.

Benchmark of a vision-language-action model (SmolVLA) against two imitation-learning policies, ACT and Diffusion Policy, on LIBERO-Object with LeRobot, all on a 6 GB RTX 3060 laptop GPU. Before training anything I reproduced the published SmolVLA checkpoint: it first scored 56% against a published 96%, and I traced the gap to a MuJoCo rendering change (pinning 3.3.2 restored it) and to open-loop action chunks (replanning every 10 actions brought it to the published level). I then trained one ACT model (52M) and one Diffusion Policy model (89M) on the same 454 demonstrations for all ten tasks, chose every setting on a separate dev split, and scored each policy on 300 held-out episodes (three seeds, re-seeded per episode so every policy faces identical episodes). Image augmentation lifted ACT from 74 to 99/100; over 300 episodes ACT reached 99.0%, ahead of SmolVLA at 92.3% and Diffusion Policy at 92.0% (paired McNemar, p < 0.001 for both). Two evaluation lessons: Diffusion Policy samples noise at inference and scored 81, 96 and 99 on the three seeds, so one seed is not enough; and GPU kernel autotuning made repeat runs differ by about 2 episodes in 100, so I made evaluation deterministic and re-measured everything, with repeat runs now matching episode for episode. A demo-count study shows how much data the recipe needs: 10 demonstrations per task give 74%, 20 give 91% and all ~45 give 99% on the same held-out episodes, and with little data the failures move from tall bottles to small, partly hidden boxes.

Built with LeRobot, PyTorch, SmolVLA, ACT, Diffusion Policy, LIBERO, MuJoCo, mixed-precision training and Python.

Post-training of the vision-language planner from One Brain, Many Bodies: a 3B model reads a camera frame with numbered markers and calls grasp, place and done tools to carry out table-top commands on the Unitree G1. I built a benchmark of six task families (bin, two objects in the bin, next to, left or right side, absent object, goal already met) where success is read from the physics state, never from what the model says, with every task feasible by a measured reach map. A privileged expert that reads the true state solves 94 of 100 held-out tasks; the stock model solves 23. QLoRA SFT (4-bit NF4, rank 16, loss on the answer tokens only) on 2432 expert steps brought it to 83, and one DAgger round, where the planner drives and the expert labels the states it reaches, brought it to 92 with failed tool calls down from 418 to 28 (paired McNemar: 61 to 1, p = 3e-17, and 11 to 2, p = 0.022). GRPO with physics rewards then solved exactly the same tasks: 66% of groups had four identical rewards, and the few errors left are rare states that random rollouts barely visit, so on this benchmark the on-policy correction that worked was DAgger. Engineering that changed the numbers: cutting the prompt from 4.5k to 1.3k tokens so training fits in 6 GB, capping the free-text thought so constrained JSON always closes, and finding a stale MuJoCo renderer that blinded the first DAgger collection.

Built with PyTorch, Hugging Face Transformers, PEFT, QLoRA, bitsandbytes, constrained JSON decoding, DAgger, GRPO, MuJoCo, Qwen2.5-VL and Python.

Industrial visual inspection on all 15 MVTec AD categories, learned from good parts only. I compared PatchCore, PaDiM, EfficientAD and zero-shot WinCLIP with my own metric code, after finding that anomalib by default tunes its thresholds on the test set. PatchCore reaches 98.8 image AUROC and 93.0 AUPRO (paper 99.0 / 93.5); an inspection region that scores the part and not the background fixed toothbrush, where a loose bristle on the backdrop had dropped it to 91.7 (now 100). I benchmarked eleven runtimes on the full test set: TensorRT fp16 runs in 5.2 ms with no accuracy loss, after I found ONNX Runtime silently falling back to the CPU because two CUDA libraries were missing. Naive INT8 took capsule from 97 to 64: the memory bank held FP32 features while the quantised backbone produced slightly different ones, and rebuilding the bank from INT8 features brought it back to 97 at 61 ms. A label-free gate decides per category whether INT8 ships (screw stays FP32). Thresholds set on training images flagged 47% of good leather, because the test parts come from another capture session; calibrating on a few good parts from the line cuts that to 3%, so the service has a calibrate endpoint and a drift monitor. The service is FastAPI with no torch inside, Prometheus metrics, and GitHub Actions CI that builds the Docker image and sends it a real request.

Built with PyTorch, anomalib, ONNX, ONNX Runtime, TensorRT, OpenVINO, NNCF INT8 quantisation, FastAPI, Docker, Prometheus, GitHub Actions and Python.

The copilot answers from the UR3e user manual in English and German, the UR Error Codes Directory (parsed into 1222 codes), the ROS 2, MoveIt 2 and UR driver docs, and the config files and 38 logged trials of my thesis. Retrieval is BM25 plus bge-m3 embeddings, fused and reranked by a multilingual cross-encoder on the CPU (right page in the top five for 99% of questions). qwen2.5:7b answers from numbered passages and can call four tools: document search, exact error code lookup, read-only SQL over the trial logs, and parameter lookup. I wrote 133 questions plus a held-out set of 49 that was never used to choose anything, graded by fact matching with no LLM judge. Five rounds of reading every failure took it from 66% to 96%, 92% on held-out. Each fix came from a real failure: the model copied example answers from its prompt, offering tools up front made it refuse questions its passages answered (so tools now come only after a refusal), it read a range written in brackets in the driver docs as a citation, and it answered questions about other robots with UR3e values. A guard drops any citation the model was not given. Plain RAG without tools reaches 82%, since it cannot run SQL over the trials. Every request goes to an audit log with the passages the model saw, so an answer can be traced to its page.

Built with Ollama, qwen2.5, bge-m3, sentence-transformers cross-encoders, rank-bm25, PyMuPDF, SQLite, FastAPI, pytest, GitHub Actions and Python.