By examining many cases, we suspect that there is a direct connection between prime numbers and difference sets. It appears that whenever n is a power of a single prime number, then a difference set containing n+1 elements can be built. Whenever n has two different prime divisors, no difference set with n+1 elements can be built.