As you look closer at the census, it quickly becomes clear. Jacob, was born in Venezuela to a father from St. Thomas and a mother from New York. Emily was also born in Venezuela to a father from South Carolina and a mother from Holland. In addition their servants were 2 from Ireland and 1 from St. Croix. What an incredible upbringing these children must have had with such an international heritage.
The records of this family can be found in The Jews of the Americas.