sending files over TCP

Please support our C# advertiser: Intel Parallel Studio Home
Thread Solved

Join Date: Mar 2008
Posts: 328
Reputation: Diamonddrake is a jewel in the rough Diamonddrake is a jewel in the rough Diamonddrake is a jewel in the rough 
Solved Threads: 39
Diamonddrake's Avatar
Diamonddrake Diamonddrake is offline Offline
Posting Whiz

sending files over TCP

 
0
  #1
Oct 9th, 2009
I recently started messing with tcp socket programming. I hacked up a little instant messaging application that seems to work has some little problems but the world isn't in need of more IM clients. but I wanted to know a good practice for sending large files over tcp sockets.

i made a simple client server app pair that will let the client send messages to the server. I have the server check the messages for special strings to tell it how to respond, if "#FILE" is sent it knows to write the next stream byte array to disk as a file. and this works great for a 25kb image. but as for a large file. how is this done?

also, how would progress data be sent?

i currently only know how to send the fully byte array in a single chuck, and I can't seem to find any GOOD, documented code examples.

if anyone knows and good links or code examples I would appreciate it. I would like to create a a simple app that runs on the system tray that would let me and my friends easily transfer files of any size back and fourth. there are a BILLION google result titles "C# sending files with TCP." but I looked at hundreds, the were either poorly written with no understandable comments, or they were other people asking my question with no answers.
Reply With Quote Quick reply to this message  
Join Date: Oct 2008
Posts: 1,948
Reputation: ddanbe has much to be proud of ddanbe has much to be proud of ddanbe has much to be proud of ddanbe has much to be proud of ddanbe has much to be proud of ddanbe has much to be proud of ddanbe has much to be proud of ddanbe has much to be proud of ddanbe has much to be proud of 
Solved Threads: 280
ddanbe's Avatar
ddanbe ddanbe is offline Offline
Posting Virtuoso
 
2
  #2
Oct 9th, 2009
I have this book at home maybe it helps.
Today is a gift, that's why it is called "The Present".
Make love, no war. Cave ab homine unius libri.
Danny
Reply With Quote Quick reply to this message  
Join Date: Mar 2008
Posts: 328
Reputation: Diamonddrake is a jewel in the rough Diamonddrake is a jewel in the rough Diamonddrake is a jewel in the rough 
Solved Threads: 39
Diamonddrake's Avatar
Diamonddrake Diamonddrake is offline Offline
Posting Whiz
 
0
  #3
Oct 9th, 2009
Thanks danny! That's a great link. nothing like a free book on the subject!. I read through a bit. i will delve into it as soon as I get a chance.
Reply With Quote Quick reply to this message  
Join Date: Oct 2008
Posts: 1,948
Reputation: ddanbe has much to be proud of ddanbe has much to be proud of ddanbe has much to be proud of ddanbe has much to be proud of ddanbe has much to be proud of ddanbe has much to be proud of ddanbe has much to be proud of ddanbe has much to be proud of ddanbe has much to be proud of 
Solved Threads: 280
ddanbe's Avatar
ddanbe ddanbe is offline Offline
Posting Virtuoso
 
0
  #4
Oct 9th, 2009
Here and there perhaps a bit outdated I intend to something with it myself. If I find the time
Today is a gift, that's why it is called "The Present".
Make love, no war. Cave ab homine unius libri.
Danny
Reply With Quote Quick reply to this message  
Join Date: Feb 2009
Posts: 3,219
Reputation: sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of 
Solved Threads: 573
Sponsor
sknake's Avatar
sknake sknake is offline Offline
.NET Enthusiast
 
0
  #5
Oct 10th, 2009
There are two ways of going about this too. FTP as an example implements both.

FTP Active -- You connect to the FTP on :21 and have a "command" session. When you request a file download or send it opens up another port and sends that information to the client. The client then connects to that port and gets/sends the file.

FTP Passive -- You connect to the FTP on :21 and when you request a file the "command" session sends or receives the file. This doesn't require two connections but also does not allow you to move around the FTP site while files are processing since the command connection is tied up with file xfer'ing.

The first method is ideal but sometimes due to router & networking issues the ports may not be set up properly so you can switch to passive to get the data. I'm not sure if you even have a command connection to begin with but I just wanted to point that out in case it makes a difference
Scott Knake
Custom Software Development
Apex Software, Inc.
Reply With Quote Quick reply to this message  
Join Date: Mar 2008
Posts: 328
Reputation: Diamonddrake is a jewel in the rough Diamonddrake is a jewel in the rough Diamonddrake is a jewel in the rough 
Solved Threads: 39
Diamonddrake's Avatar
Diamonddrake Diamonddrake is offline Offline
Posting Whiz
 
0
  #6
Oct 10th, 2009
not sure about using ftp. I don't know about creating a ftp server, I was curious about how to send messages and files simultaneously I assumed I would have to write a second thread that listened on another port. but I am using TCP and communicating via network streams. Both my clients and servers are desktop applications.
Reply With Quote Quick reply to this message  
Join Date: Feb 2009
Posts: 3,219
Reputation: sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of 
Solved Threads: 573
Sponsor
sknake's Avatar
sknake sknake is offline Offline
.NET Enthusiast
 
0
  #7
Oct 10th, 2009
Well thats what I was getting at. If you want to send messages that is identical to the "control port" over FTP, and the files over the secondary ports. The problem is if you use one port then while a file is sending you cannot send messages -- or else the message text would cause the file to get corrupted. If you use 2 ports then you can send & receive files and messages on different sockets and life is good EXCEPT when firewalls don't have port forwarding set up properly which is a very common problem.

You have to choose which way you want it to work, or try to support both. This is the same issue as FTP active/passive Today you will notice that with most instant messengers everyone connects to a central server which routes all of the messages. Way back when ICQ used to do p2p where you would connect to the other persons computer to send them an IM. As people started implementing firewalls this became very problematic so it is almost standard to use a central operating server for no purpose other than to switch packets.
Last edited by sknake; Oct 10th, 2009 at 4:29 pm.
Scott Knake
Custom Software Development
Apex Software, Inc.
Reply With Quote Quick reply to this message  
Join Date: Mar 2008
Posts: 328
Reputation: Diamonddrake is a jewel in the rough Diamonddrake is a jewel in the rough Diamonddrake is a jewel in the rough 
Solved Threads: 39
Diamonddrake's Avatar
Diamonddrake Diamonddrake is offline Offline
Posting Whiz
 
0
  #8
Oct 10th, 2009
first, Danny I read that entire book today, its full of good stuff. but it didn't exactly answer my question, because when it got to the complicated stuff such as sending files it moved over into concept, just telling you that you need to close the stream to tell the server the file send is over, and have the file sent in a separate thread and not actually going into any details. where are most of the book explains conventions of sending messages and getting connection information. regardless I still know much more about socket programming then I did previously, for this I thank you.

and as to you scott. I have my home network set up for 2 of my systems to have static IPs and my router forwards the port I am using to one of my static machines that I am using for my server. (I also use that machine for torrents, just on a different port) the code I am currently using all the clients connect to that machine. (I have a much worse issue now that my DSL IP address changes about 3 times a day so I am constantly updating it in my client code while testing)

but that is trivial to the current problem at hand.
I can't seem to send data in parts and reassembly it.
I can only achieve sending a small piece of data in entirety.

ex:
  1. //netStream is a network stream object from a tcpClient connection
  2. //this sends the entire byte array all at once.
  3. byte[] outStream = System.Text.Encoding.ASCII.GetBytes("some text to send");
  4. netStream.Write(outStream, 0, outStream.Length);
  5. netStream.Flush();
  6.  
  7. //server side
  8. //clientSocket here is a TCPClient object
  9. //fills bytearray with stream data in one read
  10.  
  11. byte[] bytesFrom = new byte[10240];//10k byte array
  12.  
  13. networkStream.Read(bytesFrom, 0, (int)clientSocket.ReceiveBufferSize);
the data here is text but I would like to send a file
I would like to know how I could send the data in like 10kb(or larger) chunks and recieve it at the server and put it back together.
Last edited by Diamonddrake; Oct 10th, 2009 at 7:25 pm.
Reply With Quote Quick reply to this message  
Join Date: Feb 2009
Posts: 3,219
Reputation: sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of 
Solved Threads: 573
Sponsor
sknake's Avatar
sknake sknake is offline Offline
.NET Enthusiast
 
1
  #9
Oct 10th, 2009
Regardless of the size of the packet you want to send it can only be as large as the MTU of your NIC
As seen:
sk@sk:~$ /sbin/ifconfig eth0 | grep -i mtu
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
Running 1500 is standard. The next most common is <1500, the least common is >1500 (called jumbo frames). The purpose of explaining all of that was to hopefully show that the chunk size doesn't really matter -- let the OS handle it.

Here is the client code:
  1. using System;
  2. using System.Collections.Generic;
  3. using System.Linq;
  4. using System.Text;
  5. using System.Net;
  6. using System.Net.Sockets;
  7. using System.IO;
  8. using System.Runtime.Remoting.Messaging;
  9.  
  10. namespace Client
  11. {
  12. public delegate void OnSendFileCompleteDelegate(object sender, SendFileCompleteEventArgs e);
  13. public class SendFileCompleteEventArgs : EventArgs
  14. {
  15. private string _fileName;
  16. private DateTime _started;
  17. private DateTime _completed;
  18. public string FileName { get { return _fileName; } }
  19. public DateTime Started
  20. {
  21. get { return _started; }
  22. internal set { _started = value; }
  23. }
  24. public DateTime Completed
  25. {
  26. get { return _completed; }
  27. internal set { _completed = value; }
  28. }
  29. public TimeSpan TimeTaken
  30. {
  31. get
  32. {
  33. return this.Completed.Subtract(this.Started);
  34. }
  35. }
  36.  
  37. private SendFileCompleteEventArgs()
  38. : this(string.Empty)
  39. {
  40. }
  41. public SendFileCompleteEventArgs(string FileName)
  42. {
  43. this._fileName = FileName;
  44. }
  45. }
  46.  
  47. public class ClientStuff
  48. {
  49. public const int DEFAULT_SERVER_PORT = 30043;
  50. private IPEndPoint endPoint;
  51.  
  52. public event OnSendFileCompleteDelegate OnSendFileComplete;
  53.  
  54. private ClientStuff()
  55. {
  56. }
  57. public ClientStuff(string IP, int Port)
  58. : this(IPAddress.Parse(IP), Port)
  59. {
  60. }
  61. public ClientStuff(IPAddress IP, int Port)
  62. : this(new IPEndPoint(IP, Port))
  63. {
  64. }
  65. public ClientStuff(IPEndPoint EndPoint)
  66. : this()
  67. {
  68. if (EndPoint == null)
  69. throw new ArgumentNullException("EndPoint");
  70.  
  71. this.endPoint = EndPoint;
  72. }
  73. public void SendFile(string FileName)
  74. {
  75. new Func<string, SendFileCompleteEventArgs>(SendFileWorker).BeginInvoke(
  76. FileName,
  77. new AsyncCallback(SendFileCallback),
  78. null);
  79. }
  80.  
  81. private SendFileCompleteEventArgs SendFileWorker(string FileName)
  82. {
  83. SendFileCompleteEventArgs result = null;
  84.  
  85. using (FileStream fs = new FileStream(FileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
  86. {
  87. using (TcpClient cli = new TcpClient())
  88. {
  89. cli.Connect(this.endPoint);
  90. result = new SendFileCompleteEventArgs(FileName);
  91. result.Started = DateTime.Now;
  92.  
  93. using (NetworkStream ns = cli.GetStream())
  94. {
  95. StreamHelper.CopyStreamToStream(fs, ns, null);
  96. ns.Flush();
  97. ns.Close();
  98. }
  99. result.Completed = DateTime.Now;
  100. return result;
  101. }
  102. }
  103. }
  104. private void SendFileCallback(IAsyncResult ar)
  105. {
  106. AsyncResult result = (AsyncResult)ar;
  107. Func<string, SendFileCompleteEventArgs> del = (Func<string, SendFileCompleteEventArgs>)result.AsyncDelegate;
  108. try
  109. {
  110. SendFileCompleteEventArgs args = del.EndInvoke(ar);
  111. if (args != null)
  112. {
  113. var evt = this.OnSendFileComplete;
  114. if (evt != null)
  115. evt(this, args);
  116. }
  117. }
  118. catch
  119. {
  120. throw; //handle this
  121. }
  122. }
  123. }
  124. }

Client caller:
  1. using System;
  2. using System.Collections.Generic;
  3. using System.ComponentModel;
  4. using System.Data;
  5. using System.Drawing;
  6. using System.Linq;
  7. using System.Text;
  8. using System.Windows.Forms;
  9. using System.Net;
  10.  
  11. namespace Client
  12. {
  13. public partial class frmClient : Form
  14. {
  15. private ClientStuff client;
  16.  
  17. public frmClient()
  18. {
  19. InitializeComponent();
  20. }
  21.  
  22. private void frmClient_Load(object sender, EventArgs e)
  23. {
  24. StartClient();
  25. this.Location = new Point(0, 0);
  26. }
  27.  
  28. private void StartClient()
  29. {
  30. if (client != null)
  31. throw new InvalidOperationException("Client is already running");
  32.  
  33. client = new ClientStuff(IPAddress.Loopback, ClientStuff.DEFAULT_SERVER_PORT);
  34. client.OnSendFileComplete += new OnSendFileCompleteDelegate(client_OnSendFileComplete);
  35. }
  36.  
  37. private void StopClient()
  38. {
  39. //I didn't add stop support for the client
  40. if (client != null)
  41. {
  42. client = null;
  43. }
  44. }
  45.  
  46. void client_OnSendFileComplete(object sender, SendFileCompleteEventArgs e)
  47. {
  48. if (this.InvokeRequired)
  49. {
  50. this.Invoke(new OnSendFileCompleteDelegate(client_OnSendFileComplete), sender, e);
  51. }
  52. else
  53. {
  54. MessageBox.Show(
  55. string.Format("Sent file {0} in {1}", e.FileName, e.TimeTaken),
  56. this.Text,
  57. MessageBoxButtons.OK,
  58. MessageBoxIcon.Information);
  59. }
  60. }
  61.  
  62. private void button1_Click(object sender, EventArgs e)
  63. {
  64. if (openFileDialog1.ShowDialog() == DialogResult.OK)
  65. {
  66. client.SendFile(openFileDialog1.FileName);
  67. }
  68. }
  69.  
  70. }
  71. }

Server Code:
  1. using System;
  2. using System.Collections.Generic;
  3. using System.Linq;
  4. using System.Text;
  5. using System.Net.Sockets;
  6. using System.Net;
  7. using System.IO;
  8. using System.Threading;
  9. using System.Runtime.Remoting.Messaging;
  10.  
  11. namespace Server
  12. {
  13. #region event stuff
  14. public delegate void OnFileReceivedDelegate(object sender, OnFileReceivedArgs e);
  15. public class OnFileReceivedArgs : EventArgs
  16. {
  17. private byte[] _buffer;
  18. public byte[] Buffer { get { return _buffer; } }
  19.  
  20. private OnFileReceivedArgs()
  21. : base()
  22. {
  23. }
  24. public OnFileReceivedArgs(byte[] buffer)
  25. : this()
  26. {
  27. this._buffer = buffer;
  28. }
  29. }
  30. #endregion
  31.  
  32. public class ServerStuff : IDisposable
  33. {
  34. public const int DEFAULT_SERVER_PORT = 30043;
  35.  
  36. private TcpListener listener;
  37. private List<WeakReference> connections; //dont want our handy conn list to stop garbage collection
  38.  
  39. public event OnFileReceivedDelegate OnFileReceived;
  40.  
  41. public int ConnectionCount
  42. {
  43. get
  44. {
  45. lock (connections)
  46. {
  47. return connections.Count;
  48. }
  49. }
  50. }
  51.  
  52. private ServerStuff()
  53. : this(DEFAULT_SERVER_PORT)
  54. {
  55. }
  56. public ServerStuff(int Port)
  57. {
  58. connections = new List<WeakReference>();
  59. listener = new TcpListener(IPAddress.Any, DEFAULT_SERVER_PORT);
  60. }
  61.  
  62. public void Start()
  63. {
  64. listener.Start();
  65. listener.BeginAcceptSocket(new AsyncCallback(ConnectionCallback), listener);
  66. }
  67. public void Stop()
  68. {
  69. listener.Stop();
  70. KillRunningThreads();
  71. }
  72.  
  73. private void TrackThread()
  74. {
  75. lock (connections)
  76. {
  77. connections.Add(new WeakReference(Thread.CurrentThread, false));
  78. }
  79. }
  80. private void RemoveThread()
  81. {
  82. Thread t = Thread.CurrentThread;
  83. lock (connections)
  84. {
  85. for (int i1 = 0; i1 < connections.Count; i1++)
  86. {
  87. if ((connections[i1].Target as Thread) == t)
  88. {
  89. connections.RemoveAt(i1);
  90. break;
  91. }
  92. }
  93. }
  94. }
  95. private Thread[] GetRunningThreads()
  96. {
  97. lock (connections)
  98. {
  99. List<Thread> result = new List<Thread>();
  100.  
  101. foreach (WeakReference wr in connections)
  102. {
  103. if (wr.IsAlive)
  104. result.Add((Thread)wr.Target);
  105. }
  106.  
  107. return result.ToArray();
  108. }
  109. }
  110.  
  111. private void KillRunningThreads()
  112. {
  113. Thread[] threads = GetRunningThreads();
  114. foreach (Thread t in threads)
  115. {
  116. try
  117. {
  118. t.Abort();
  119. }
  120. catch { }
  121. }
  122. }
  123.  
  124. private void ConnectionCallback(IAsyncResult ar)
  125. {
  126. TcpListener listener = (TcpListener)ar.AsyncState;
  127. try
  128. {
  129. Socket s = listener.EndAcceptSocket(ar);
  130. new Func<Socket, byte[]>(HandleSocketComms).BeginInvoke(
  131. s,
  132. new AsyncCallback(HandleSocketCommsCallback),
  133. s);
  134. }
  135. catch
  136. {
  137. //You should handle this but this should be a _rare_ error
  138. throw;
  139. }
  140. finally
  141. {
  142. //Prime up the listener to accept another inbound request
  143. listener.BeginAcceptSocket(new AsyncCallback(ConnectionCallback), listener);
  144. }
  145. }
  146.  
  147. /// <summary>
  148. /// This basically blocks until the socket closes and we have
  149. /// received all of the data
  150. /// </summary>
  151. /// <param name="s"></param>
  152. /// <returns></returns>
  153. private byte[] HandleSocketComms(Socket s)
  154. {
  155. //Dont do any exception handling here
  156. TrackThread();
  157.  
  158. using (MemoryStream ms = new MemoryStream()) //This may be a bad idea for big files :)
  159. {
  160. do
  161. {
  162. int szToRead = s.Available;
  163. byte[] buffer = new byte[szToRead];
  164.  
  165. int szRead = s.Receive(buffer, szToRead, SocketFlags.None);
  166. if (szRead > 0)
  167. {
  168. ms.Write(buffer, 0, szRead);
  169. Console.WriteLine("Read " + szRead.ToString());
  170. }
  171. }
  172. while (SocketConnected(s));
  173.  
  174. return ms.ToArray();
  175. }
  176. //return result;
  177. }
  178.  
  179. private static bool SocketConnected(Socket s)
  180. {
  181. return !(s.Poll(1000, SelectMode.SelectRead) && (s.Available == 0));
  182. }
  183.  
  184. private void HandleSocketCommsCallback(IAsyncResult ar)
  185. {
  186. byte[] file = null;
  187. bool error = false;
  188.  
  189. try
  190. {
  191. AsyncResult result = (AsyncResult)ar;
  192. var del = (Func<Socket, byte[]>)result.AsyncDelegate;
  193. file = del.EndInvoke(ar);
  194.  
  195. Socket s = (ar.AsyncState as Socket);
  196. if (s != null)
  197. {
  198. //Sockets throw exceptions++ when disposed
  199. try
  200. {
  201. s.Close(0);
  202. ((IDisposable)s).Dispose();
  203. }
  204. catch { }
  205. }
  206. }
  207. catch (ThreadAbortException) //When we stop our server
  208. {
  209. //We were killed! Just let it die out
  210. error = true;
  211. }
  212. catch
  213. {
  214. //Might want to handle this
  215. error = true;
  216. }
  217. finally
  218. {
  219. RemoveThread(); //conn is closed at this point
  220. if (!error && (file != null) && (file.Length > 0))
  221. {
  222. var del = this.OnFileReceived;
  223. if (del != null)
  224. del(this, new OnFileReceivedArgs(file));
  225. }
  226. }
  227. }
  228.  
  229. #region IDisposable Members
  230. public void Dispose()
  231. {
  232. if (listener != null)
  233. {
  234. listener.Stop();
  235. listener = null;
  236. }
  237. KillRunningThreads();
  238. }
  239. #endregion
  240. }
  241. }

Server caller:
  1. using System;
  2. using System.Collections.Generic;
  3. using System.ComponentModel;
  4. using System.Data;
  5. using System.Drawing;
  6. using System.Linq;
  7. using System.Text;
  8. using System.Windows.Forms;
  9. using System.IO;
  10. using System.Diagnostics;
  11.  
  12. namespace Server
  13. {
  14. public partial class frmServer : Form
  15. {
  16. private ServerStuff server;
  17.  
  18. public frmServer()
  19. {
  20. InitializeComponent();
  21. }
  22.  
  23. private void StartServer()
  24. {
  25. if (server != null)
  26. throw new InvalidOperationException("Server is already running");
  27.  
  28. server = new ServerStuff(ServerStuff.DEFAULT_SERVER_PORT);
  29. server.Start();
  30. server.OnFileReceived += new OnFileReceivedDelegate(server_OnFileReceived);
  31. }
  32.  
  33. private void StopServer()
  34. {
  35. if (server != null)
  36. {
  37. server.OnFileReceived -= new OnFileReceivedDelegate(server_OnFileReceived);
  38. server.Stop();
  39. ((IDisposable)server).Dispose();
  40. server = null;
  41. }
  42. }
  43.  
  44. void server_OnFileReceived(object sender, OnFileReceivedArgs e)
  45. {
  46. if (this.InvokeRequired)
  47. {
  48. //Just in case we want to play with the UI
  49. this.Invoke(new OnFileReceivedDelegate(server_OnFileReceived), sender, e);
  50. }
  51. else
  52. {
  53. string tmpFile = Path.GetTempFileName();
  54. #region change the extension
  55. {
  56. string newFile = Path.ChangeExtension(tmpFile, ".bmp");
  57. File.Move(tmpFile, newFile);
  58. tmpFile = newFile;
  59. }
  60. #endregion
  61.  
  62. File.WriteAllBytes(tmpFile, e.Buffer);
  63. Process p = Process.Start(tmpFile);
  64. p.WaitForExit();
  65. File.Delete(tmpFile);
  66. }
  67. }
  68.  
  69.  
  70.  
  71. private void frmServer_Load(object sender, EventArgs e)
  72. {
  73. StartServer();
  74. this.Location = new Point(SystemInformation.WorkingArea.Width - this.Width, 0);
  75. }
  76.  
  77. private void button1_Click(object sender, EventArgs e)
  78. {
  79. //byte[] b = File.ReadAllBytes(@"C:\img\dw\desktop.bmp");
  80. //System.Diagnostics.Debugger.Break();
  81. //6912054
  82. }
  83. }
  84. }

I didn't implement full stop functionality but I partially plumbed it and the KillThreads() code is untested but it should work.
Attached Files
File Type: zip daniweb.tcp.zip (31.8 KB, 16 views)
Scott Knake
Custom Software Development
Apex Software, Inc.
Reply With Quote Quick reply to this message  
Join Date: Feb 2009
Posts: 3,219
Reputation: sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of sknake has much to be proud of 
Solved Threads: 573
Sponsor
sknake's Avatar
sknake sknake is offline Offline
.NET Enthusiast
 
0
  #10
Oct 10th, 2009
A little more on frame size:

The standard MTU for computers today is 1500 as mentioned earlier. The TCP Header size is 32 bytes so you can really send (1500-32)=1468 bytes of data in a packet. The header contains the packet's routing information.

By decreasing your MTU you can force the number of packets up. If you wanted to send 2900 bytes of data:
1500MTU : 2900/(1500-32)=1.97547683 (2 packets total, round up)
750 MTU: 2900/(750-32)=4.0389972 (5 packets total, round up)
What is the advantage of forcing the packet count up? Traffic Shaping!
If you want to limit bandwidth to X Mbps you have more fine-grain control by holding up a smaller packet than a larger packet. So by increasing the count and decreasing the payload it gives you more control to regulate your total throughput.

The problems: Going back to the fact that most computers operate with 1500 MTU if your MTU is 750 and you go a website, say, www.google.com and their MTU is 1500 you will have a problem:
1500byte google packet -> 750 MTU machine. Your machine can't receive the packet so it says "Wait, That is too big!" and it will send Google back an ICMP fragmentation-needed message. Google will then fragment the packet to the size specified in the ICMP message and send it back in a size that you can read.

Now that should work, right? No. Let me quote the iptables man page:
TCPMSS
This target allows to alter the MSS value of TCP SYN packets, to control the maximum size for that connection (usually limiting it to your outgoing interface's MTU minus 40). Of course, it can only be used in conjunction
with -p tcp. It is only valid in the mangle table.
This target is used to overcome criminally braindead ISPs or servers which block ICMP Fragmentation Needed packets. The symptoms of this problem are that everything works fine from your Linux firewall/router, but machines
behind it can never exchange large packets:
1) Web browsers connect, then hang with no data received.
2) Small mail works fine, but large emails hang.
3) ssh works fine, but scp hangs after initial handshaking.
Workaround: activate this option and add a rule to your firewall configuration like:
iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN \
-j TCPMSS --clamp-mss-to-pmtu
So if you're running 750 and google is running 1500 and someone in the middle of you two drops the ICMP Fragmentation packet then you're screwed. You won't be able to communicate with the remote server. You can set up your edge device to clamp outbound packets to the MTU discovery by Path MTU Discovery (PMTU) but this also does not work all of the time as expected.

Likewise all of this works in reverse for jumbo frames BUT if you have a larger MTU than sites on the internet it won't be a problem:
1500MTU Google >> 50000000MTU machine will make your machine laugh and say "give me more data"

I have read that some people claim you can communicate with LAN appliances better with jumbo frames BUT the biggest bottleneck in TCP/IP is usually disk I/O speed on either end. 99% of the time someone screws with an MTU it breaks something.

You should be ready to start writing network applications now

[edit]
My math was off. The total TCP header size can be up to 40 bytes, not just 32. That is insignificant in the points made though.
[/edit]
Last edited by sknake; Oct 10th, 2009 at 10:00 pm.
Scott Knake
Custom Software Development
Apex Software, Inc.
Reply With Quote Quick reply to this message  
Reply

Tags
networking, sockets, tcp

This thread has been marked solved.
Perhaps start a new thread instead?
Message:


Thread Tools Search this Thread



About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC