Tuesday, October 20, 2009

Fair Use Statement

I just want to take a moment here to point out something.

Fair Use Statement

This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. Such material is made available in an effort to advance understanding of issues of environmental and humanitarian significance. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes.
Posted by at 11:45 AM 0 comments

Monday, October 19, 2009

My notes on TCP/IP stacks.

Please excuse, some if this is just stuff I grabed off of web sites for further investigation.

TCP/IP stacks


Mini TCP/IP stacks.

uIP   http://www.sics.se/~adam/uip/index.php/Main_Page

lwIP  http://savannah.nongnu.org/projects/lwip/    lwIP is significantly more complex

uC/IP http://sourceforge.net/projects/ucip/   uC/IP - TCP/IP for microcontrollers

Alpine - user space TCP/IP  (have source code somewhere)
UPDATE July 20, 2011:
I found my Alpine code, but it's only the Alpine4Linux. 

The file can be downloaded from : http://www.dnull.com/Alpine/
I'd like to put these up on source forge or at least something like box.net where the code can no longer get lost. 

Daytona ""   http://nms.lcs.mit.edu/~kandula/data/daytona.pdf

----------------


> i want to move tcp/ip stack(including routing and
> netfilter) to userspace, my goal is to trace all the
> instructions involved in a firewall and router since i
> don't know how to trace these instructions inside the
> kernel. i want to get something like:
>
> incoming ip packets(a file)-->fake ISR-->tcp/ip
> stack-->outgoing ip packets( to /dev/null).
>
> my question is: is it possible and relatively easy to
> move tcp/ip stack to user space?
This comes up fairly frequently, it might be a good addition to the FAQ.
Here's my attempt at an answer culled from prior messages.

Several people have user-mode network stacks at various levels of
development, but it is *highly* unlikely for them ever to get into
the kernel proper (see the monolithic versus microkernel debate at
http://www.kernel.org/pub/linux/docs/lkml/#s15-4).

Here are some URLs to which you can refer for more information:
        http://www.cl.cam.ac.uk/Research/SRG/netos/arsenic/
        http://www.cs.nwu.edu/~pdinda/minet/minet.html
        http://www.joerch.org/tcpip/
        http://freshmeat.net/projects/libutcp/

However, for security purposes, you probably do not want a user-mode stack.
You want an extensible packet handling mechanism, and can be found with:
        iptables/ipchains -- the native Linux firewalling tools,
                http://netfilter.samba.org/
        tc -- the Traffic control program,
                http://www.sparre.dk/pub/linux/tc/
        libpcap -- packet capture library,
                http://www.tcpdump.org
------------------------------
---------------------
I've started a syncookie fw using a daemon process  ???

I do this using the REDIRECT ( ipchains ) / QUEUE ( ipfilter ) targets,
to get the packets to userspace. Once there, you can do what you want
using libpcap or syuscalls.

-------------------------------------------------------

[10] C.A. Thekkath, T.D. Nguyen, E.Moy and E.D.
Lazowska, ‘Implementing Network Protocols at User
Level,’ in Proceedings SIGCOMM 1993,
San Francisco, pp. 64-73, September, 1993.

[13] C. Maeda and B.N. Bershad, ‘Protocol service
decomposition for high-performance networking,’ in
14th ACM Symposium on Operating Systems
Principles, December 1993.

------------------------------------------------------
[15] G.L. Peterson, “Myths about the mutual exclusion
problem,’ in Information Processing Letters, Vol. 12,
No. 3, pp. 115-116, June 1981.
A Appendices
A.1 Peterson’s algorithm
Peterson’s algorithm for mutual exclusion between two
processes is as follows (also see [15]):
Lock()
{
wantAccess[ME]=TRUE;
nextAccess=YOU;
while (wantAccess[YOU] && nextAccess==YOU)
/* Add yield code here */
;
}
Release()
{
wantAccess[ME]=0;
}

A.2 Edwards’ Algorithm
This algorithm for mutual exclusion exploits the fact that
the child process runs at real-time priority and so cannot be
pre-empted by the user process:
Lock()
{
#ifdef CHILD_PROCESS
if (disable_child) {
/* If user process has disabled us
* then we block and wait for him to
* wake us up
*/
child_blocking=TRUE;
WaitOnSemaphoreForWakeup();
}
#else
/* We know child must be blocked at the
* moment (thus not holding lock) as it
* runs as a real-time priority so can
* not be pre-empted
*/
disable_child=TRUE;
#endif
}
Release()
{
#ifdef CHILD_PROCESS
if (wakeup_user_process) {
WakeupSleepingUserProcess();
}
#else
disable_child=FALSE;
if (child_blocking) {
child_blocking=FALSE;
SignalWaitSemaphore();
}
#endif
}

 -------------------------------------------
http://www.vzavenue.net/~neelnatu/alpine4linux/alpine4linux.txt
 
UPDATE July 20, 2011:
I found my Alpine code, but it's only the Alpine4Linux. 

The file can be downloaded from : http://www.dnull.com/Alpine/


--------------------------------------------------


Date: Fri, 30 May 2003 11:53:23 -0700 (PDT)
To: freebsd-net@freebsd.org
natun@rpi.edu or neelnatu@yahoo.com

Neelkanth Natu (neelnatu@yahoo.com)

Hi all,

Alpine4Linux is a userlevel FreeBSD 4.8 networking stack running on top of a stock Linux kernel.  It is an implementation of an idea that I came across in a paper[1] by David Ely, Stefan Savage and David Wetherall.

Alpine4Linux consists of a userland server program that runs the FreeBSD kernel code as well as
the unmodified networking stack. Client programs use the Alpine stack by setting the LD_PRELOAD
environment variable to link to libraries, that intercept socket related system calls. These
intercepted system calls are routed to the Alpine server over a TCP connection established on the loopback interface.

Alpine4Linux goes great lengths to ensure that its behavior is identical to that of a FreeBSD
kernel as far as networking is concerned. Thus, in addition to the unmodified FreeBSD stack.
Alpine4Linux also has unmodified socket layer code, file descriptor code, tsleep and wakeup for
e.g. It also has a rich client-side library that supports almost all socket functions as well as
functions like fork() that are commonly used by server programs.

I have tested stock Linux programs like telnet, nmap, ifconfig, vsftpd-1.1.3 etc. against
the Alpine stack. I have only compiled it on a Redhat 8.x Linux box, so if someone manages to run it on another Linux distribution/kernel please let me know.

You can find more information at: http://www.vzavenue.net/~neelnatu/alpine4linux

Hopefully someone finds use for this.

thanks
Neel

[1] Alpine: A user-level infrastructure for network protocol development
    David Ely, Stefan Savage, David Wetherall
    http://alpine.cs.washington.edu/

----------------------------------------------------------


Application-L evel Protocol Infrastructure for Network Experimentation
alpine_debug.c
Alpine_FreeBSD3x-0.10.tar.gz.
http://web.archive.org/web/20050208075748/http://alpine.cs.washington.edu/distfiles/Alpine_FreeBSD3x-0.10.tar.gz

-----------------------

--------------
TCP_congestion_avoidance_algorithm

http://www.cs.arizona.edu/projects/protocols/
http://en.wikipedia.org/wiki/TCP_Vegas
http://en.wikipedia.org/wiki/TCP_congestion_avoidance_algorithm
 
TCP Tahoe and Reno
TCP Vegas
TCP New Reno
TCP Hybla   high-latency terrestrial or satellite radio link
TCP BIC     In linux 2.6.8 through 2.6.18
TCP CUBIC  In linux  since version 2.6.19
Compound TCP (CTCP) is a Microsoft algorithm that is part of the Windows Vista and Window Server 2008 TCP
YeAH TCP - Yet Another Highspeed TCP - Linux

2.6.22
TCP Illinois congestion control. This is an implementation of TCP Illinois invented by Shao Liu at University of Illinois. It is a another variant of Reno which adapts the alpha and beta parameters based on RTT. The basic idea is to increase window less rapidly as delay approaches the maximum. See the papers and talks to get a more complete description

YeAH-TCP congestion control algorithm implementation. YeAH-TCP is a sender-side high-speed enabled TCP congestion control algorithm, which uses a mixed loss/delay approach to compute the congestion window. It's design goals target high efficiency, internal, RTT and Reno fairness, resilience to link loss while keeping network elements load as low as possible. For further details look here: http://wil.cs.caltech.edu/pfldnet2007/paper/YeAH_TCP.pdf

TCP cubic update for 2.6.22. The new version improves on its scalability, fairness and stability. So in all properties, we confirmed it shows better performance


Fastsoft
http://www.fastsoft.com/downloads/FASTTCP-IEEENetwork-0501.pdf

-----------------------------------

http://www.sics.se/~adam/miniweb/

web-enabled embedded systems

Many devices (printers, etc.) now use not just standard Internet Protocol but have a small web server embedded in them for input and to indicate status rather than using floppy disks or LCD panels or tiny little keyboards or other proprietary communication protocols.
[FIXME: Is there a better file for me to keep this category ?]
web appliances
[ Intelligent Instrumentation, Inc. http://www.instrument.com/ethernet data acquisition system ] is very similar ...



---------------------------------------

Fixing the Unfairness of TCP Congestion Control

"George Ou, Technical Director of ZDNet, has an analysis today of an engineering proposal to address congestion issues on the internet. It's an interesting read, with sections such as "The politicization of an engineering problem" and "Dismantling the dogma of flow rate fairness". Short and long term answers are suggested, along with some examples of what incentives it might take to get this to work. Whichever side of the neutrality debate you're on, this is worth consideration."

By 1999, the first P2P (peer-to-peer) application called Swarmcast began to blatantly exploit Jacobson’s TCP congestion control mechanism.  Using a technique called “parallel incremental downloading”, Swarmcast could grab a much larger share of the pie at the expense of others by exploiting the multi-stream and persistence loophole.  These two loopholes would be used by every P2P application since.







Posted by at 5:53 AM 7 comments
Labels: ,

Mobile-ITX new super small PC motherboard form factor

This is a new standard PC motherboard form factor that is almost the same size as a cell phone.


http://www.linuxdevices.com/news/NS2010384636.html
 

Via CEO Wenchi Chen revealed a business card-sized motherboard billed as the "world's first industry-standard form-factor for PC/phone convergence," at Computex today. The "mobile-ITX" board measures 3 x 1.8 inches -- half the size of Via's "pico-ITX" form-factor -- and runs Linux or Windows XP Embedded.
 
Via's mobile-ITX board prototype
(Click to enlarge)

The mobile-ITX board that Chen demonstrated this morning appears to be based on a 1GHz "C7-S" processor -- apparently a standard Via C7-M shoe-horned into a 9 x 11mm package. The chip had not previously been announced. The mobile-ITX board also apparently uses an "S" (small) version of the CX700 integrated north-/south-bridge chipset. And, it appears to have an on-board DC-DC converter. Additionally, according to Via, the board includes a CDMA baseband processor chip, suggesting that the mobile-ITX board could be used as the basis for x86-compatible smartphones.

Motorola phone compared to mobile-ITX

According to a brief item at EpiaCenter, Via's mobile-ITX board will be available with 256MB or 512MB of RAM soldered on-board, and will run Linux or Windows XP Embedded. Even an embedded version of Windows Vista may be too much for the little board, however, a Via spokesperson admits.

Chen stated, "This prototype Mobile-ITX gives a glimpse into the future of ultra mobile devices and the real convergence of computing and communications. Yet this is only the start. We see the platform shrinking still further, with ever richer blends of functionality, that will truly make ultra mobility the normal way of enjoying our content and our communications."

Marketing Manager Tim Brown with Via's NanoBook UMPC reference design, and CEO Chen with mobile-ITX prototype
(Click to enlarge)


Given that Chen positioned mobile-ITX as an "industry standard," lots of technical specifications about the form-factor should become available shortly. Meanwhile, EpiaCenter has posted a few photos, and a video, here. 

Posted by at 5:37 AM 6 comments

make depends

Dependency Management
The irony of Makehttp://www.ddj.com/linux-open-source/184406479




Advanced Auto-Dependency Generation
http://make.paulandlesley.org/autodep.html
Posted by at 5:27 AM 0 comments

The Linux Kernel API

Documentation on many of the kernel API calls, with some searching I Found some excellent pages on this.

http://www.gnugeneration.com/books/linux/2.6.20/kernel-api/

http://tldp.org/LDP/lkmpg/2.6/html/index.html

http://kernelnewbies.org/KernelHacking

http://www.advancedlinuxprogramming.com/alp-folder

Linux USB drivers — Understanding and developing Linux USB drivers
Embedded Linux optimizations — Optimizing the Linux kernel and applications for speed, size, RAM, power and cost.
http://free-electrons.com/articles

Advanced Linux Programming
http://en.wikibooks.org/wiki/Linux_kernel
Posted by at 5:25 AM 0 comments

millisecond tick in C++

To compile run
g++ -l pthread -o millisecondtick millisecondtick.cpp

Turned out I had to use one static member function to jump off onto another to get up and running.

/* millisecondtick.cpp
 * Creates Millisecond Tick 
 */

#include 
#include 
#include 

#include "millisecondtick.h"

MilliSecond_Tick::MilliSecond_Tick(){
   Running = FALSE;
};



void* MilliSecond_Tick::runx(void *pMilliSecond_Tick){

    ((MilliSecond_Tick*)pMilliSecond_Tick)->run(NULL);
}



int MilliSecond_Tick::start( pFnVoid func ){

pFnVoidVoid  prun;

 
  prun = runx;
 
   if (MilliSecond_Tick::Running != FALSE){
     printf("Error MilliSecond_Tick already started, can not start second instance\n");
     return 0 ;
   }

  callback = func;
  pthread_create(&tid, NULL, prun , this);
    Running = TRUE;

  return 0;
}




void MilliSecond_Tick::stop(){
MilliSecond_Tick::Running = FALSE;
pthread_join(tid, NULL);

}


void MilliSecond_Tick::tick(){
  (*callback)(); 
}


void* MilliSecond_Tick::run(void * nothing){

    struct sched_param sparam;

    sparam.sched_priority = sched_get_priority_max(SCHED_FIFO);
    if (pthread_setschedparam(pthread_self(), SCHED_FIFO, &sparam) == 0) {
                        // success, this application has sufficient privileges
    }
    else {
            // setting priority failed, probably due to insufficient privieges
    }

    for (;;) {                           // for-ever loop of the ticker thread
        static const struct timespec nanowait = {0, 0*1000*1000};    // 1 ms
        nanosleep(&nanowait, (struct timespec *)0);

     if (Running == FALSE){
        break;
 }

        tick();
    }
}



void tickss( ){
 static int x = 0, y = 0;
x++;
  if (x > 1000){
     x = 0;
     y++;
     printf(">%d\n",y);
     } 
}



int main(){

 MilliSecond_Tick cmt;

 cmt.start(&tickss );

 sleep(10);
 cmt.stop();

}

// millisecondtick.h

#include 

#define TRUE 1
#define FALSE 0 

typedef void(*pFnVoid)();
typedef void*(*pFnVoidVoid)( void *);

class MilliSecond_Tick
{
  private:
    int  Running;
    pFnVoid callback; 
    pthread_t tid;
    static void *runx(void *pMilliSecond_Tick);
    void tick();
  public:

  MilliSecond_Tick(); 
  ~MilliSecond_Tick(){};
  int start( pFnVoid func ) ;
  void stop(); 
  void * run(void * nothing);
};
Posted by at 5:15 AM 0 comments
Labels:
Subscribe to: Posts (Atom)