Both uses start from the same plain picture — people called small — but they point in different directions, and neither grows out of the other. In one, small means small in power: the people at the bottom of a country or a company, the ones a decision lands on. In the other, small means small in body and mostly hidden: the fairy folk of Irish stories, named this way instead of being called fairies outright. It helps to read the phrase as 'the small folk' and then let the situation say which small folk is meant.
可以想成：兩個意思都從同一個簡單的畫面出發——被叫做「小」的一群人。但它們往兩個方向走，誰也不是從誰長出來的。一個方向的「小」是權力上的小：一個國家或一家公司最底層的人，決定一下來，最先被壓到的就是他們。另一個方向的「小」是身形上的小，而且多半藏起來：愛爾蘭故事裡的仙族，大家不直接叫 fairy，就叫他們 the little people。可以先把這個詞讀成「小小的那群人」，再讓上下文告訴你是哪一種小。