this post was submitted on 27 Jun 2023
9 points (100.0% liked)

Stable Diffusion

5027 readers
3 users here now

Discuss matters related to our favourite AI Art generation technology

Also see

Other communities

founded 2 years ago
MODERATORS
db0@lemmy.dbzer0.com
Even_Adder@lemmy.dbzer0.com
9
[Solved] Help working with an extremely narrowly trained model (poptalk.scrubbles.tech)
submitted 2 years ago* (last edited 2 years ago) by scrubbles@poptalk.scrubbles.tech to c/stable_diffusion@lemmy.dbzer0.com
3 comments fedilink hide all child comments
 

Hey folks, first time posting here, I hope this is the right place. I'm having a frustrating issue, I downloaded a Lycoris and added the extension to SD, and it works great, but was clearly only trained on close up portraits.

As soon as my prompt includes anything outside of the shoulders it switches to someone else. I tried training my own Lora with more poses to be more varied, but my face is distored now.

Can I combine these, either in prompts, or can I combine the models somehow? Could I use img2img on only the portrait area? (I tried but I don't know how to focus on just the head area, like "Apply this lycoris to the (masked) head"

Thanks!

[EDIT]: I figured it out, I was using whole picture and didn't understand how the "Only masked area" worked. I played with it a lot and figured out I could create a base image and have a wonky face, then in img2img mask the face and then bring in the lycoris, use only masked, and it'd fill it perfectly

you are viewing a single comment's thread
view the rest of the comments
[–] nodsocket@lemmy.world 3 points 2 years ago (1 children)

Maybe use one model to do the body, then use inpaint on the face using the portrait model?

[–] scrubbles@poptalk.scrubbles.tech 2 points 2 years ago (1 children)

How do you target just the in that scenario? I've tried to do that (just the mask, mask over the face, then do whole image and swap out the prompt to the lycoris one), but it gives me the same issue as using the prompt, an unrelated face. The lycoris only seems to work if I do just a straight up portrait.

Is there a way to "select" part of the image only for the inpaint?

[–] nodsocket@lemmy.world 2 points 2 years ago* (last edited 2 years ago)

I'm not really sure. Normally I would just play with the denoising until it looks good, that's all I know.

I've heard of ControlNet, which lets you control aspects of the image better than without, using things like specific poses or gestures as an additional input. It might be worth looking into that.