American Samoa is part of the United States but American Samoa Immigration is not. They operate outside the of the national system to a degree (as well as American Samoa customs). For most people, this does not matter as the basic laws apply in American Samoa.
It is possible to be legally in American Samoa and not legally in the United States. It is also true that having a legal status on the island will not get you access to the rest of the United States. This is a common misconception that many have. However, it has benefited the economy of the US territory.
What you do need to know is that as citizens of the United States, you have all the rights that you would in the United States as far as travel. American Samoa Immigration allows you to stay on the island as long as you want, you can work on the island if you need. The only thing you can’t do is own land.
What if you are not American?
There is a list of waiver free is basically the same as for the rest of the United States. Freely Associated States and close allies are allowed to come to the island with very little trouble. Many New Zealanders live in American Samoa for example.
If you are not from one of them and you need a visa, American Samoa Immigration requires that you have a sponsor to live there but visiting is pretty easy. This would not be a concern for most people coming from the developed world.
I have never heard of anyone traveling to American Samoa having much of a problem. If you do, just go to the American Samoa Immigration office in the government complex downtown and they will work things out.
What is your experience with traveling to American Samoa?