Constellation: AI Material Explorer
ComfyUIPythonConstellation combines a generative material engine with a 2D exploration space to create a source of inspiration for CMF and industrial designers. Designers can input a guiding theme, generate swatches, combine them, and build 3D assets from AI generated materials.
Constellation
A library of footwear materials, each synthesized by a procedural-plus-generative pipeline, arranged by visual similarity into a space you can explore.
What if you could fully generate novel materials with AI, then explore them in 2D space?
Technical Details
Constellation is a hybrid material-generation pipeline: procedural code controls structure; AI provides material realism.
A custom Python engine generates the structural foundation for every material: seven parametric pattern families — triangular, hexagonal, diamond, voronoi, perforated, weave, and ribbed — built on NumPy, each driven by a shared parameter contract of scale, density, regularity, rotation, and seed.
Every pattern tiles seamlessly and exports as a heightfield and an edge-control map, giving the diffusion stage clean structural conditioning to build on. Keeping structure procedural rather than generated means it stays fully art-directable — I control the geometry precisely, and let the AI interpret the surface.
Each structure is rendered into a photorealistic material through Stable Diffusion XL, conditioned by ControlNet so the generated surface follows the underlying geometry. Prompts are assembled from a weighted vocabulary matrix — material × finish × quality × context — producing controlled variation across hundreds of specimens from a compact set of terms.
ControlNet strength is deliberately tuned below full lock, so the model interprets the structure rather than tracing it — the balance point where the geometry guides but the AI still contributes material realism.
To make the library navigable, every generated material is embedded with a CLIP vision model, encoding each swatch as a high-dimensional visual fingerprint. Those embeddings are projected into two dimensions through dimensionality reduction, so materials that look alike land near each other — the arrangement emerges from visual similarity itself, not manual tagging.
The result is a latent space you can wander, where neighboring specimens share tonal and material character, and the gaps between clusters become the interesting territory.
Selecting two materials generates the space between them by interpolating their underlying recipes: structural parameters blend numerically toward a true midpoint, while the material vocabulary cross-inherits from both parents.
Because structure is procedural and continuous, the in-between is genuinely computable — not a visual dissolve, but a new specimen sharing measurable traits of each parent. It's the core idea of the tool in miniature: the designer's taste selects the direction, and the system generates toward it.
The natural next phase is taking selected materials off the flat swatch and onto form. Every specimen already carries a complete recipe — structural parameters plus material definition — which makes it fully reproducible as a physically-based material: displacement and normal detail derived from the procedural heightfield, surface response driven by the generated swatch.
Batch-rendered across a curated set in a production DCC, these become the payoff — explored materials realized as tactile, lit surfaces on a footwear form, closing the loop from generative exploration to production-ready output.