Using ControlNet-like Models (Edges, Depth)

Fater includes powerful AI models that can be guided by Control Layers, specifically Edges maps and Depth maps. These "Controlled" models allow you to dictate the structure, composition, and form of your generated images with much greater precision than using text prompts alone.

This guide focuses on the workflow within the Image Editor.

When to Use Controlled Models

  • Precise Structural Generation: When you need the AI-generated image to closely follow specific outlines, shapes, or 3D forms (e.g., architectural designs, product outlines, character poses).

  • Compositional Control: To define the layout and placement of elements in a scene.

  • Re-styling with Structure Preservation: Apply a new artistic style (via prompt) while maintaining the underlying structure defined by an Edges or Depth map.

Workflow for Using Controlled Models

  1. Prepare Your Control Layer:

    • First, you need either an Edges map or a Depth map. (Refer to Working with Control Layers - Step 1 for details on creating or uploading these).

      • Edges Map: Generate from existing canvas content using the CPU icon on the Inpaint Toolbar, draw manually, or upload.

      • Depth Map: Generate from existing canvas content using the Mountain icon on the Inpaint Toolbar, or upload.

    • Ensure this Control Layer image is present in your Layers Tab.

  2. Designate as Control Layer & Set Type:

    • In the Layers Tab, drag your Edges/Depth map layer into the dedicated "Control Layers" section (this section appears when a "Controlled" AI model is selected).

    • Within the "Control Layers" section, make sure the "Preprocessed Type" dropdown for that layer is set correctly:

      • edges for an Edges map.

      • depth for a Depth map.

  3. Select a "Controlled" AI Model:

    • Go to the Model Selector in the Left Sidebar.

    • Choose the "Generate" or "Edit" category tab.

    • Select the "Controlled" SubCategory.

    • In the "Control Type" dropdown, select:

      • "Contour" if you are using an Edges map.

      • "Depth" if you are using a Depth map.

    • Finally, choose a user-facing model name from the "Model" dropdown (e.g., "High [Generate/Controlled/Contour]").

  4. Configure Model-Specific Parameters:

    • The Left Sidebar will display parameters for the selected Controlled model. Key parameters often include:

      • Prompt & Negative Prompt: Describe the content, style, and details you want the AI to generate while adhering to the Control Layer's structure.

      • Control Strength / Adherence: (Name varies) This is crucial. It determines how strongly the AI prioritizes the Control Layer's guidance versus the text prompt's creative direction.

      • Steps, Guidance, Seed: Standard generation parameters.

  5. Set the Generation Area Bounding Box:

    • Ensure the dashed bounding box on your canvas covers the area where you expect the generation to occur, relative to your Control Layer.

  6. Ensure Control Layer Visibility:

    • In the Layers Tab, make sure your designated Control Layer (the Edges or Depth map) is visible (Eye icon is on). Only visible Control Layers are used.

  7. Initiate Generation:

    • Click the Generate button (✨ Sparkles icon) in the Floating Prompt Area.

  8. Review and Use Results:

    • The AI will generate an image, attempting to match your prompt's description while conforming to the structure of your Edges or Depth map.

    • The first result is automatically added as a new layer.

    • Use the Generation Task List to view and select other variations if multiple were generated.

Example Scenario : Visualizing a Building from an Edges Sketch

  1. Control Layer: You have a simple line drawing (Edges map) of a building's facade on a layer, and you've designated it as an "edges" Control Layer.

  2. Model: Select a model like "High [Generate/Controlled/Contour]".

  3. Parameters: Adjust Control Strength to be relatively high.

  4. Prompt: "Modern sustainable office building, glass and reclaimed wood facade, lush green rooftop garden, sunny day, photorealistic."

  5. Generate.

  6. The AI should generate a photorealistic building that follows the structural lines of your Edges sketch.

Controlled models offer a significant leap in guiding AI generation, allowing you to blend textual creativity with precise visual structure. Experiment with different Control Layers, prompts, and strength parameters to achieve your desired outcomes.

PreviousImage-to-Image & InpaintingNextUpscaling Images

Last updated