There are a lot of components that affect the file transfer speeds over a network and it is just not just only the network speed. The bottleneck is usually going to be the storage because disk have a maximum transfer rate that could be lower than you network speed. For example a 1Gb network would give you a rate 0f 125mbps, this mean that a 1Gb file would take close to 8 seconds to be transfer between 2 nodes. Now if your node or server has a mechanical drive that gives you around 50mpbs max, That means that 1Gb transfer file would take around 3 minutes instead of 8 seconds.
Let me give you a couple of recommendations if you want to maximize file transfer between servers.
- If you have 10Gb infrastructure make use of it. The transfer rate is around 1.25Gbps
- Do not use the onboard NICs. The reason being is that also affects the transfer speeds. Use the PCI Express expansion card on the node or server.
- You can change settings on the NIC to improve the transfer rate such as buffer size, RSS(Only works with multiple Nics), use Jumbo frames (for this to work properly the settings needs to be set on the switch), hard code the speed on the NIC. I would recommend SolarFlare network cards.
- Add more NICs so parallel file transfer can be utilize
- Windows 2012 R2 navitely can use RDMA (Remote Direct Memory Access) . You may need to do some configuration on the network.
- You want to use a modern operation system like Windows 2012 R2. The reason being is take advantage on the SMB 3.0 protocol. You can read this blog for more information here
Now if you need to calculate how long would it take to transfer a file I have written a small function to calculate based on 1Gb, 10GB and 40Gb network.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| function Get-TransferTime | |
| { | |
| [CmdletBinding()] | |
| Param | |
| ( | |
| [Long] | |
| $FileSizeGb, | |
| $FileSizeTb | |
| ) | |
| Begin { | |
| [decimal]$1Gb = 0.125 | |
| [decimal]$1Mb = 0.0125 | |
| [long]$10Gb = 1.250 | |
| [int]$40Gb = 5 | |
| [long]$100Gb = 12.5 | |
| } | |
| Process { | |
| if($FileSizeTb) { | |
| [decimal]$FileSize = $FileSizeTb * 1000 | |
| } | |
| else { | |
| [decimal]$FileSize = $FileSizeGb | |
| } | |
| $Time1Gb = $FileSize / $1Gb | |
| $Time10Gb = $FileSize / $10Gb | |
| $Time40Gb = $FileSize / $40Gb | |
| $Time100Gb = $FileSize / $100Gb | |
| $Time1Gb = New-TimeSpan -Seconds $Time1Gb | |
| $Time10Gb = New-TimeSpan -Seconds $Time10Gb | |
| $Time40Gb = New-TimeSpan -Seconds $Time40Gb | |
| $Time100Gb = New-TimeSpan -Seconds $Time100Gb | |
| $Time1Gb = $Time1Gb | Select-Object days, hours, minutes, seconds | |
| $Time10Gb = $Time10Gb | Select-Object days, hours, minutes, seconds | |
| $Time40Gb = $Time40Gb | Select-Object days, hours, minutes, seconds | |
| } | |
| End { | |
| Write-Output "`n To Transfer $FileSize GB over a 1GB network, will take $Time1Gb" | |
| Write-Output "`n To Transfer $FileSize GB over a 10GB network, will take $Time10Gb" | |
| Write-Output "`n To Transfer $FileSize GB over a 40GB network, will take $Time40Gb" | |
| } | |
| } |
Now lets do a couple of examples
How long would it take 1GB to transfer from one node to another on perfect conditions
Get-TransferTime -FileSizeGb 1
1Gb Network -> 8 seconds
10Gb Network-> 1 second
40Gb Network -> less than 1 second
10Gb FILE TRANSFER
1Gb Network -> 1 minute and 20 seconds
10Gb Network-> 10 seconds
40Gb Network -> 2 seconds
1TB FILE TRANSFER
Get-TransferTime -FileSizeTb 1
1Gb Network -> 2 hours and 13 minutes
10Gb Network-> 16 minutes
40Gb Network -> 3 minutes and 20 seconds
10TB FILE TRANSFER
Get-TransferTime -FileSizeTb 10
1Gb Network -> 22 hours and 13 minutes
10Gb Network-> 2 hours and 46 minutes
40Gb Network -> 33 minutes and 20 seconds
The key thing here is that you need to pay attention to the whole infrastructure to maximize transfer speed. Keep in mind if you want to have faster, and secure file transfers is super important to use modern operating systems like Windows 2012 R2 and use enhanced protocols like SMB 3.X
Gabe's Blog 