Not all flash drives are created equal.
In general the more memory you have the slower it is.
Think about 10 shoeboxes in a row going away from you on your left and 100 on your right.
You know in the 7th box on your left is a 100$ bill, and in the 70th box on your right there is 100$ bill.
Which one could u get to faster? (hint: the one w/ less storage)
That said, spinning harddrives kinda went against this due to the fact that they made data more dense so it was quicker to traverse. Going w/ the previous analogy, imagine the same 10 boxes on your left w/ the Franklin in the 7th box, but on your right there is only 1 box that has a divider in it that divides it into 100 squares w/ the Bill still in the 70th square, whats quicker going to the 7th shoebox on ur left or reaching into the 70th partition of the 1 box on your right?
For flash dives the first analogy still holds true except that the differences in speed based on size are greatly outweighed but other factors like the bus architecture and memory controller used by the drive.