How come "black" refers to the whole *****!d race...?

"yellow" (even though it isn't said as much as black and white for some reason) refers to the whole mongoloid race, but "white" refers to only Europeans and culture is involved as to determining who's white or not?