Kalosm Vision

Kalosm Vision is a collection of image models and utilities for the Kalosm framework. It includes utilities for generating images from text and segmenting images into objects.

Image Generation

You can use the Wuerstchen model to generate images from text:

use futures_util::StreamExt;
use kalosm_vision::{Wuerstchen, WuerstchenInferenceSettings};

#[tokio::main]
async fn main() {
    let model = Wuerstchen::builder().build().await.unwrap();
    let settings = WuerstchenInferenceSettings::new(
        "a cute cat with a hat in a room covered with fur with incredible detail",
    );

    if let Ok(mut images) = model.run(settings) {
        while let Some(image) = images.next().await {
            if let Some(buf) = image.generated_image() {
                buf.save(&format!("{}.png", image.sample_num())).unwrap();
            }
        }
    }
}

Image Segmentation

Kalosm supports image segmentation with the SegmentAnything model. You can use the SegmentAnything::segment_everything method to segment an image into objects or the SegmentAnything::segment_from_points method to segment an image into objects at specific points:

use kalosm::vision::*;

let model = SegmentAnything::builder().build().unwrap();
let image = image::open("examples/landscape.jpg").unwrap();
let x = image.width() / 2;
let y = image.height() / 4;
let images = model
    .segment_from_points(
        SegmentAnythingInferenceSettings::new(image)
            .unwrap()
            .add_goal_point(x, y),
    )
    .unwrap();

images.save("out.png").unwrap();