yeemachine /kalidokit

Blendshape and kinematics calculator for Mediapipe/Tensorflow.js Face, Eyes, Pose, and Finger tracking models.

★ 5,730⑂ 721👁 78⑃ 21● 0▤ 0
TypeScriptMITCreated 2021-10-08kalidoface.com
+1
日增长
5,730
stars
721
forks
78
watchers
21
prs
0
issues
0
releases

README

[![NPM Package][npm]][npm-url]
[![NPM Bundle Size][minimized-size]][npm-url]
[![jsDelivr hits (npm)][js-delivr]][js-delivr-url]
[![NPM Downloads][npm-downloads]][npmtrends-url]
[![Website][website]][website-url]
[![Discord Shield][discord]][discord-url]

| :exclamation: KalidoKit is now officially deprecated and solutions will be directly integrated into MediaPipe :exclamation:| 
|--------------------------------------|

 Face, Pose, and Hand Tracking Calculator

Kalidokit is a blendshape and kinematics solver for Mediapipe/Tensorflow.js face, eyes, pose, and hand tracking models, compatible with Facemesh, Blazepose, Handpose, and Holistic. It takes predicted 3D landmarks and calculates simple euler rotations and blendshape face values.

As the core to Vtuber web apps, Kalidoface and Kalidoface 3D, KalidoKit is designed specifically for rigging 3D VRM models and Live2D avatars!

NOTE: This is a JS library meant for developers using Mediapipe pretrained models and not a complete app by itself. Please visit Kalidoface for the full vtuber experience 😉!






 Install

 Via NPM


npm install kalidokit


js
import  as Kalidokit from "kalidokit";

// or only import the class you need

import { Face, Pose, Hand } from "kalidokit";


 Via CDN

js



 Methods

Kalidokit is composed of 3 classes for Face, Pose, and Hand calculations. They accept landmark outputs from models like Facemesh, Blazepose, Handpose, and Holistic.

js
// Accepts an array(468 or 478 with iris tracking) of vectors
Kalidokit.Face.solve(facelandmarkArray, {
    runtime: "tfjs", // mediapipe or tfjs
    video: HTMLVideoElement,
    imageSize: { height: 0, width: 0 },
    smoothBlink: false, // smooth left and right eye blink delays
    blinkSettings: [0.25, 0.75], // adjust upper and lower bound blink sensitivity
});

// Accepts arrays(33) of Pose keypoints and 3D Pose keypoints
Kalidokit.Pose.solve(poseWorld3DArray, poseLandmarkArray, {
    runtime: "tfjs", // mediapipe or tfjs
    video: HTMLVideoElement,
    imageSize: { height: 0, width: 0 },
    enableLegs: true,
});

// Accepts array(21) of hand landmark vectors; specify 'Right' or 'Left' side
Kalidokit.Hand.solve(handLandmarkArray, "Right");

// Using exported classes directly
Face.solve(facelandmarkArray);
Pose.solve(poseWorld3DArray, poseLandmarkArray);
Hand.solve(handLandmarkArray, "Right");


Additional Utils

js
// Stabilizes left/right blink delays + wink by providing blenshapes and head rotation
Kalidokit.Face.stabilizeBlink(
    { r: 0, l: 1 }, // left and right eye blendshape values
    headRotationY, // head rotation in radians
    {
        noWink = false, // disables winking
        maxRot = 0.5 // max head rotation in radians before interpolating obscured eyes
    });

// The internal vector math class
Kalidokit.Vector();


 VRM and Live2D Sample Code

Quick-start your Vtuber app with these simple remixable examples.

 VRM Glitch Template

Face, full-body, and hand tracking in under 350 lines of javascript. This dem

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON