TCP

Sliding window

http://www.tcpipguide.com/free/t_TCPSlidingWindowAcknowledgmentSystemForDataTranspo-3.htm

The send window is the key to the entire TCP sliding window system: it represents the maximum number of unacknowledged bytes a device is allowed to have outstanding at once. The usable window is the amount of the send window that the sender is still allowed to send at any point in time; it is equal to the size of the send window less the number of unacknowledged bytes already transmitted.

Advertisements
TCP

C++

.hpp

It’s not perfect, and you would usually resort to techniques like the Pimpl Idiom to properly separate interface and implementation, but it’s a good start.

 

A compilation in C++ is done in 2 major phases:

  1. The first is the compilation of “source” text files into binary “object” files: The CPP file is the compiled file and is compiled without any knowledge about the other CPP files (or even libraries), unless fed to it through raw declaration or header inclusion. The CPP file is usually compiled into a .OBJ or a .O “object” file.
  2. The second is the linking together of all the “object” files, and thus, the creation of the final binary file (either a library or an executable).

Where does the HPP fit in all this process?

A poor lonesome CPP file…

The compilation of each CPP file is independent from all other CPP files, which means that if A.CPP needs a symbol defined in B.CPP, like:

// A.CPP
void doSomething()
{
   doSomethingElse(); // Defined in B.CPP
}

// B.CPP
void doSomethingElse()
{
   // Etc.
}

It won’t compile because A.CPP has no way to know “doSomethingElse” exists… Unless there is a declaration in A.CPP, like:

// A.CPP
void doSomethingElse() ; // From B.CPP

void doSomething()
{
   doSomethingElse() ; // Defined in B.CPP
}

Then, if you have C.CPP which uses the same symbol, you then copy/paste the declaration…

COPY/PASTE ALERT!

Yes, there is a problem. Copy/pastes are dangerous, and difficult to maintain. Which means that it would be cool if we had some way to NOT copy/paste, and still declare the symbol… How can we do it? By the include of some text file, which is commonly suffixed by .h, .hxx, .h++ or, my preferred for C++ files, .hpp:

// B.HPP (here, we decided to declare every symbol defined in B.CPP)
void doSomethingElse() ;

// A.CPP
#include "B.HPP"

void doSomething()
{
   doSomethingElse() ; // Defined in B.CPP
}

// B.CPP
#include "B.HPP"

void doSomethingElse()
{
   // Etc.
}

// C.CPP
#include "B.HPP"

void doSomethingAgain()
{
   doSomethingElse() ; // Defined in B.CPP
}

How does include work?

Including a file will, in essence, parse and then copy-paste its content in the CPP file.

For example, in the following code, with the A.HPP header:

// A.HPP
void someFunction();
void someOtherFunction();

… the source B.CPP:

// B.CPP
#include "A.HPP"

void doSomething()
{
   // Etc.
}

… will become after inclusion:

// B.CPP
void someFunction();
void someOtherFunction();

void doSomething()
{
   // Etc.
}

One small thing – why include B.HPP in B.CPP?

In the current case, this is not needed, and B.HPP has the doSomethingElse function declaration, and B.CPP has the doSomethingElse function definition (which is, by itself a declaration). But in a more general case, where B.HPP is used for declarations (and inline code), there could be no corresponding definition (for example, enums, plain structs, etc.), so the include could be needed if B.CPP uses those declaration from B.HPP. All in all, it is “good taste” for a source to include by default its header.

Conclusion

The header file is thus necessary, because the C++ compiler is unable to search for symbol declarations alone, and thus, you must help it by including those declarations.

One last word: You should put header guards around the content of your HPP files, to be sure multiple inclusions won’t break anything, but all in all, I believe the main reason for existence of HPP files is explained above.

 

C++

wireshark

last time using this was long time ago which is why I almost forgot everything about it.

So this time I’ll write it down to help remember it.

  1. display filter & capture filter :capture filter is what packets you really capture and display is what you want to see. We can use the configuration icon on the top left for fast choosing capture filter, Expression icon for display filter.
  2. DSCP:
    Differentiated Services Code Point (DSCP)

    Originally defined as the Type of service (ToS) field. This field is now defined by RFC 2474 (updated by RFC 3168 and RFC 3260) forDifferentiated services (DiffServ). New technologies are emerging that require real-time data streaming and therefore make use of the DSCP field. An example is Voice over IP (VoIP), which is used for interactive data voice exchange.   https://en.wikipedia.org/wiki/IPv4#DSCP                     DiffServ uses a 6-bit differentiated services code point (DSCP) in the 8-bit differentiated services field (DS field) in the IP header for packet classification purposes. The DS field and ECN field replace the outdated IPv4 TOS field

  3. IntServ specifies a fine-grained QoS system, which is often contrasted with DiffServ’s coarse-grained control system. In order for IntServ to work, all routers along the traffic path must support it. Furthermore, many states must be stored in each router. As a result, IntServ works on a small-scale, but as you scale up to a system the size of the Internet, it is difficult to keep track of all of the reservations.[1]
  4. Per-hop behaviour

    From Wikipedia, the free encyclopedia

    In computer networking, per-hop behaviour (PHB) is a term used in differentiated services (DiffServ) or multiprotocol label switching(MPLS). It defines the policy and priority applied to a packet when traversing a hop (such as a router) in a DiffServ network.

    Rule that governs how packets are handled within a diffserv[1] network is called the Per-Hop Behavior (PHB). PHBs are defined to support the general properties controlled by IP precedence. DSCP Contains 6-bits, PHBs are created (one for each combination of the top 3 bits) of the form bbb000 to match the precedence behaviors and leaves the other DSCP values open where each b may take the value zero or 1.

    DSCP Bit Settings Meaning
    000000 Best effort
    bbb000 Conforms to the requirements of Type of Service queuing precedence
    bbbbb0 Available for standardization
    bbbb11 For experimental of local network usage
    bbbb01 For experimental of local network usage, but may be taken for standardization
    Class selector values
    DSCP Binary Hex Decimal Typical application Examples
    CS0 (Default) 000 000 0x00 0
    CS1 001 000 0x08 8 Scavenger YouTube, Gaming, P2P
    CS2 010 000 0x10 16 OAM SNMP,SSH,Syslog
    CS3 011 000 0x18 24 Signaling SCCP,SIP,H.323
    CS4 100 000 0x20 32 Realtime TelePresence
    CS5 101 000 0x28 40 Broadcast video Cisco IPVS
    CS6 110 000 0x30 48 Network control EIGRP,OSPF,HSRP,IKE
    CS7 111 000 0x38 56

     

  5. List of IP protocol numbers

    From Wikipedia, the free encyclopedia

    This is a list of IP numbers used in the Protocol field of the IPv4 header and the Next Header field of IPv6 header.

    Decimal Hex Keyword Protocol References
    0 0x00 HOPOPT IPv6 Hop-by-Hop Option RFC 2460
    1 0x01 ICMP Internet Control Message Protocol RFC 792
    2 0x02 IGMP Internet Group Management Protocol RFC 1112
    3 0x03 GGP Gateway-to-Gateway Protocol RFC 823
    4 0x04 IP-in-IP IP in IP (encapsulation) RFC 2003
    5 0x05 ST Internet Stream Protocol RFC 1190, RFC 1819
    6 0x06 TCP Transmission Control Protocol RFC 793
    7 0x07 CBT Core-based trees RFC 2189
    8 0x08 EGP Exterior Gateway Protocol RFC 888
    9 0x09 IGP Interior Gateway Protocol (any private interior gateway (used by Cisco for their IGRP))
    10 0x0A BBN-RCC-MON BBN RCC Monitoring
    11 0x0B NVP-II Network Voice Protocol RFC 741
    12 0x0C PUP Xerox PUP
    13 0x0D ARGUS ARGUS
    14 0x0E EMCON EMCON
    15 0x0F XNET Cross Net Debugger IEN 158
    16 0x10 CHAOS Chaos
    17 0x11 UDP User Datagram Protocol RFC 768
    18 0x12 MUX Multiplexing IEN 90
    19 0x13 DCN-MEAS DCN Measurement Subsystems
    20 0x14 HMP Host Monitoring Protocol RFC 869
    21 0x15 PRM Packet Radio Measurement
    22 0x16 XNS-IDP XEROX NS IDP
    23 0x17 TRUNK-1 Trunk-1
    24 0x18 TRUNK-2 Trunk-2
    25 0x19 LEAF-1 Leaf-1
    26 0x1A LEAF-2 Leaf-2
    27 0x1B RDP Reliable Datagram Protocol RFC 908
    28 0x1C IRTP Internet Reliable Transaction Protocol RFC 938
    29 0x1D ISO-TP4 ISO Transport Protocol Class 4 RFC 905
    30 0x1E NETBLT Bulk Data Transfer Protocol RFC 998
    31 0x1F MFE-NSP MFE Network Services Protocol
    32 0x20 MERIT-INP MERIT Internodal Protocol
    33 0x21 DCCP Datagram Congestion Control Protocol RFC 4340
    34 0x22 3PC Third Party Connect Protocol
    35 0x23 IDPR Inter-Domain Policy Routing Protocol RFC 1479
    36 0x24 XTP Xpress Transport Protocol
    37 0x25 DDP Datagram Delivery Protocol
    38 0x26 IDPR-CMTP IDPR Control Message Transport Protocol
    39 0x27 TP++ TP++ Transport Protocol
    40 0x28 IL IL Transport Protocol
    41 0x29 IPv6 IPv6 Encapsulation RFC 2473
    42 0x2A SDRP Source Demand Routing Protocol RFC 1940
    43 0x2B IPv6-Route Routing Header for IPv6 RFC 2460
    44 0x2C IPv6-Frag Fragment Header for IPv6 RFC 2460
    45 0x2D IDRP Inter-Domain Routing Protocol
    46 0x2E RSVP Resource Reservation Protocol RFC 2205
    47 0x2F GRE Generic Routing Encapsulation RFC 2784, RFC 2890
    48 0x30 MHRP Mobile Host Routing Protocol
    49 0x31 BNA BNA
    50 0x32 ESP Encapsulating Security Payload RFC 4303
    51 0x33 AH Authentication Header RFC 4302
    52 0x34 I-NLSP Integrated Net Layer Security Protocol TUBA
    53 0x35 SWIPE SwIPe IP with Encryption
    54 0x36 NARP NBMA Address Resolution Protocol RFC 1735
    55 0x37 MOBILE IP Mobility (Min Encap) RFC 2004
    56 0x38 TLSP Transport Layer Security Protocol (using Kryptonet key management)
    57 0x39 SKIP Simple Key-Management for Internet Protocol RFC 2356
    58 0x3A IPv6-ICMP ICMP for IPv6 RFC 4443, RFC 4884
    59 0x3B IPv6-NoNxt No Next Header for IPv6 RFC 2460
    60 0x3C IPv6-Opts Destination Options for IPv6 RFC 2460
    61 0x3D Any host internal protocol
    62 0x3E CFTP CFTP
    63 0x3F Any local network
    64 0x40 SAT-EXPAK SATNET and Backroom EXPAK
    65 0x41 KRYPTOLAN Kryptolan
    66 0x42 RVD MIT Remote Virtual Disk Protocol
    67 0x43 IPPC Internet Pluribus Packet Core
    68 0x44 Any distributed file system
    69 0x45 SAT-MON SATNET Monitoring
    70 0x46 VISA VISA Protocol
    71 0x47 IPCU Internet Packet Core Utility
    72 0x48 CPNX Computer Protocol Network Executive
    73 0x49 CPHB Computer Protocol Heart Beat
    74 0x4A WSN Wang Span Network
    75 0x4B PVP Packet Video Protocol
    76 0x4C BR-SAT-MON Backroom SATNET Monitoring
    77 0x4D SUN-ND SUN ND PROTOCOL-Temporary
    78 0x4E WB-MON WIDEBAND Monitoring
    79 0x4F WB-EXPAK WIDEBAND EXPAK
    80 0x50 ISO-IP International Organization for Standardization Internet Protocol
    81 0x51 VMTP Versatile Message Transaction Protocol RFC 1045
    82 0x52 SECURE-VMTP Secure Versatile Message Transaction Protocol RFC 1045
    83 0x53 VINES VINES
    84 0x54 TTP TTP
    84 0x54 IPTM Internet Protocol Traffic Manager
    85 0x55 NSFNET-IGP NSFNET-IGP
    86 0x56 DGP Dissimilar Gateway Protocol
    87 0x57 TCF TCF
    88 0x58 EIGRP EIGRP
    89 0x59 OSPF Open Shortest Path First RFC 1583
    90 0x5A Sprite-RPC Sprite RPC Protocol
    91 0x5B LARP Locus Address Resolution Protocol
    92 0x5C MTP Multicast Transport Protocol
    93 0x5D AX.25 AX.25
    94 0x5E IPIP IP-within-IP Encapsulation Protocol RFC 2003
    95 0x5F MICP Mobile Internetworking Control Protocol
    96 0x60 SCC-SP Semaphore Communications Sec. Pro
    97 0x61 ETHERIP Ethernet-within-IP Encapsulation RFC 3378
    98 0x62 ENCAP Encapsulation Header RFC 1241
    99 0x63 Any private encryption scheme
    100 0x64 GMTP GMTP
    101 0x65 IFMP Ipsilon Flow Management Protocol
    102 0x66 PNNI PNNI over IP
    103 0x67 PIM Protocol Independent Multicast
    104 0x68 ARIS IBM’s ARIS (Aggregate Route IP Switching) Protocol
    105 0x69 SCPS SCPS (Space Communications Protocol Standards) SCPS-TP[1]
    106 0x6A QNX QNX
    107 0x6B A/N Active Networks
    108 0x6C IPComp IP Payload Compression Protocol RFC 3173
    109 0x6D SNP Sitara Networks Protocol
    110 0x6E Compaq-Peer Compaq Peer Protocol
    111 0x6F IPX-in-IP IPX in IP
    112 0x70 VRRP Virtual Router Redundancy Protocol, Common Address Redundancy Protocol (not IANA assigned) VRRP:RFC 3768
    113 0x71 PGM PGM Reliable Transport Protocol RFC 3208
    114 0x72 Any 0-hop protocol
    115 0x73 L2TP Layer Two Tunneling Protocol Version 3 RFC 3931
    116 0x74 DDX D-II Data Exchange (DDX)
    117 0x75 IATP Interactive Agent Transfer Protocol
    118 0x76 STP Schedule Transfer Protocol
    119 0x77 SRP SpectraLink Radio Protocol
    120 0x78 UTI Universal Transport Interface Protocol
    121 0x79 SMP Simple Message Protocol
    122 0x7A SM Simple Multicast Protocol draft-perlman-simple-multicast-03
    123 0x7B PTP Performance Transparency Protocol
    124 0x7C IS-IS over IPv4 Intermediate System to Intermediate System (IS-IS) Protocol over IPv4 RFC 1142 and RFC 1195
    125 0x7D FIRE Flexible Intra-AS Routing Environment
    126 0x7E CRTP Combat Radio Transport Protocol
    127 0x7F CRUDP Combat Radio User Datagram
    128 0x80 SSCOPMCE Service-Specific Connection-Oriented Protocol in a Multilink and Connectionless Environment ITU-T Q.2111 (1999)
    129 0x81 IPLT
    130 0x82 SPS Secure Packet Shield
    131 0x83 PIPE Private IP Encapsulation within IP Expired I-D draft-petri-mobileip-pipe-00.txt
    132 0x84 SCTP Stream Control Transmission Protocol
    133 0x85 FC Fibre Channel
    134 0x86 RSVP-E2E-IGNORE Reservation Protocol (RSVP) End-to-End Ignore RFC 3175
    135 0x87 Mobility Header Mobility Extension Header for IPv6 RFC 6275
    136 0x88 UDPLite Lightweight User Datagram Protocol RFC 3828
    137 0x89 MPLS-in-IP Multiprotocol Label Switching Encapsulated in IP RFC 4023
    138 0x8A manet MANET Protocols RFC 5498
    139 0x8B HIP Host Identity Protocol RFC 5201
    140 0x8C Shim6 Site Multihoming by IPv6 Intermediation RFC 5533
    141 0x8D WESP Wrapped Encapsulating Security Payload RFC 5840
    142 0x8E ROHC Robust Header Compression RFC 5856
    143-252 0x8F-0xFC UNASSIGNED
    253-254 0xFD-0xFE Use for experimentation and testing RFC 3692
    255 0xFF Reserved.

Streaming Media Protocol

http://www.streamingmedia.com/Articles/Editorial/What-Is-…/What-Is-a-Streaming-Media-Protocol-84496.aspx

 

 

To follow all this, I recommend adding these fields as columns:

  • TCP length (tcp.len)
  • Sequence number (tcp.seq)
  • Next expected sequence number (tcp.nxtseq)
  • Acknowledgment number (tcp.ack)

There the follwing length now:
Frame length: Total length of the Frame, including the Padding Fields (if present and needed) of the Ethernet Layer
Captured Length: Frame Length which is captured (Interresting if a filter has been used)
IP.TotalLength: Total Packet Length. from IP-Header until Layer 7 payload ends
TCP.SegmentLegth: Resulting TCP Payload and only calculated by Wireshark
TCP.HeaderLength: Is the length of the TCP Header, because header size is variabel

Identification of TCP stream in wireshark

https://www.appneta.com/blog/how-to-easily-capture-tcp-conversation-streams/

 

TSHARK

 

https://ask.wireshark.org/questions/4677/easy-way-to-save-tcp-streams

 

 

wireshark

10.19 reading

The Billion Dollar Startup Club

http://graphics.wsj.com/billion-dollar-club/

Screenshot from 2015-10-19 10:46:17

snapchat: video or pic msg, destroyed once read.

palantir: data analysis

WeWork is an American company which provides shared workspace, community, and services for entrepreneurs, freelancers, startups and small businesses.

lufux:Shanghai Lujiazui International Financial Asset Exchange Co:peer to peer lending for startup

spotify: music

TBD

10.19 reading

netcat

http://www.binarytides.com/netcat-tutorial-for-beginners/

https://nmap.org/ncat/guide/ncat-usage.html#ncat-connect

So basically netcat is a tool to do some bidirectional network communication over the TCP/UDP protocols.

ncat <host> [<port>]

<host> may be a hostname or IP address, and <port> is a port number. Listen mode is the same, with the addition of the --listen option (or its -l alias):

ncat –listen [<host>] [<port>]
ncat -l [<host>] [<port>]

1. Telnet

The very first thing netcat can be used as is a telnet program. Lets see how.

$ nc -v google.com 80

Now netcat is connected to google.com on port 80 and its time to send some message. Lets try to fetch the index page. For this type “GET index.html HTTP/1.1” and hit the Enter key twice. Remember twice.

$ nc -v google.com 80
Connection to google.com 80 port [tcp/http] succeeded!
GET index.html HTTP/1.1

HTTP/1.1 302 Found
Location: http://www.google.com/
Cache-Control: private
Content-Type: text/html; charset=UTF-8
X-Content-Type-Options: nosniff
Date: Sat, 18 Aug 2012 06:03:04 GMT
Server: sffe
Content-Length: 219
X-XSS-Protection: 1; mode=block

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

The output from google.com has been received and echoed on the terminal.

2. Simple socket server

To open a simple socket server type in the following command.

$ nc -l -v 1234

The above command means : Netcat listen to TCP port 1234. The -v option gives verbose output for better understanding. Now from another terminal try to connect to port 1234 using telnet command as follows :

$ telnet localhost 1234
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
abc
ting tong

After connecting we send some test message like abc and ting tong to the netcat socket server. The netcat socket server will echo the data received from the telnet client.

$ nc -l -v 5555

Connection from 127.0.0.1 port 5555 [tcp/rplay] accepted
abc
ting tong

This is a complete Chatting System. Type something in netcat terminal and it will show up in telnet terminal as well. So this technique can be used for chatting between 2 machines.

netcat

monitor system

My goal is to build a cloud monitor system in one year.

Questions:

  1. the goal of MS(monitoring system)/ difference with others
  2. the architecture and  function of MS/difference with others
  3. how to realize it
  4. refinements

1. the goal

provide complete and real time info about system, including resource, workload, forecast and etc.

2. function

http://www.datacenterknowledge.com/archives/2015/10/02/how-to-select-the-right-cloud-management-tools/

  • alerts and alarms:Being able to set up alerts so that the proper administrator is notified depending on the issue is important. If a storage alert goes out to a server admin, the response may not be as fast as it would be had the alert gone out to a storage administrator.                             different alerts go to different administrator
  • resource management
  • user count
  • failover capabilities
  • roles and privileges:This isolation of roles creates effective audit trails. It also greatly reduces the risk that a team member will make the wrong changes to the system.
  • SLA considerations:This means monitoring uptime and environment usage. Depending on the type of SLA, different metrics are important to the administrator. This might mean monitoring the number of VMs working or adjusting downtime requirements
  • Testing and maintenance

samples of monitoring system

http://talkincloud.com/cloud-computing-management/20-cloud-monitoring-and-management-tools-which-are-best

http://demo.appmanager.com/showapplication.do?method=showApplications

http://www.logicmonitor.com/platform/

monitor system