It’s not the influx of foreigners but the supposedly irrational fear of foreigners that is the problem