How do i keep a pom pom from falling apart?

I'm currently trying to make a pom-pom to put on top of a beanie thats I made, but whenever I pull on one of the pieces of yarn it gets pulled out of the pom. Is there a way to keep it from doing this or do I just have to tie a really tight knot and just make sure I don't pull on it?