Model gets cloned multiple times if the scene has a crowd

I am using Flux and able to create very good scenes with multiple people or crowd. However, a lot of times the scene clones my model multiple times to the scene. So I have 2-3 people all appear the same as my model which is funny but makes the photo unusable.

For instance if my prompt has "two people sitting and talking together", a lot of times both the people are the same model.