I remember that when I was at school, I was told that to make friends, I needed to be more friendly and approach people more, and then people would respond. So one day, I sat next to a boy in class that I didn't normally talk to and said 'hi'. He promptly turned to his friends who were sitting behind him and said, 'what's she doing sitting next to me?' So much for that advice.
This is how it was for me, too. I rarely joined in groups - found them a bit intimidating/threatening, too. Part of it was fear of rejection, and part not knowing how to join in.
Another thing that happens a lot to me, is that I talk to one person, and one to one it's ok, but then another person comes along to join the conversation, and the person I'm talking to starts to talk to them and I get left out/ignored and feeling redundant. Their conversation is somehow different - it flows between them it a way that it doesn't with me, and I don't know how to make it flow like this.