Actually you will get a delay regardless of partition size as it doesn't make any difference whether it's copying on same partition, between partitions on same drive, or between partitions on different drives, it is still doing a file copy (you end up with 2 copies of same file). What you are referring to is a file move within same partition, where the file data isn't copied, just the directory structure is updated to reflect the new location of the file. However, if a file is moved between partitions on same or different drives, then the file data has to be copied and the source then deleted. So in that case a file move is the same as doing a file copy followed by a source file delete.
It actually has everything to do with I/O because the maximum data transfer rate is related to the rotational speed of the drives involved in the copying process. Regardless of whether the data is being copied on same drive or different drives, it still has to travel across the ATA bus to the computers memory and back again. As I said before, when duplicating files on same drive, the drive head(s) must seek back & forth to the source location (for reads) and the destination location (for writes) therefore involving much more drive head seeking. This is where buffering more data will help as it is increasing the data chunk size and this reduces the amount of head seeks necessary to complete the copy process.
No it isn't a long time if you do something else while copying but if you scale the filesize up, the amount of wasted time induced by head seeks caused by copying small chunks, increases and can amount to quite a lot of time. This is annoying if you need to get tasks done involving the copied file and must wait for it each time. If you need to do this many times a day the time wasted amounts to quite a lot with very big (xxGB sized) files. The only solution to this I have so far is to use 2 drives but sometimes I need to duplicate on same drive.
As I said, copying between 2 similar performing drives versus copying on either one of those drives only, is usually about a minimum of 1:2 ratio because one drive performs reads while the other writes, whereas a single drive has to do both reads and writes which it can't perform simultaneously, it has to do one or the other to achieve the task. What I don't like about windows is that it only manages about 1:3 ratio which I think is excessive and due to it copying in small chunks resulting in an increase of unecessary head seeking. This is why I would like to tweak windows to use more RAM during large file copies on same drive. It can also be helpful when copying between 2 drives where 1 or both are in use by other processes during a copy and/or 1 drive has much greater performance than the other.