Freesco add-ons

Default Freesco package comes without GNU grep and find. Existing grep is just a hard-link to a general script, 'a:' (that's it, just an a and a colon).

% # in my case, 36 is the inode number for 'a:'
% gnu_find * -printf "%i %p\n" | grep ^"36 " | awk '{print $2}'
a: backup basename clr cp-rf crontab dir dirname edit free fromdos grep hostname installpkg listpkg ll lsmod minicom netinfo passwd play ps removepkg report sysedit sysinfo telnet todos uname uptime useradd userdel useredit userlist utils

[Linux] gnu_find /bin/ -printf "%i %p\n" | grep grep$

[Linux] more /bin/grep
#!/bin/sh
#
# Don't touch next lines. This script provides:
#@@ dir edit telnet lsmod clr a: userlist ll
#@@ basename dirname hostname passwd sysinfo report
#@@ sysedit free grep ps minicom useradd useredit
#@@ userdel netinfo uptime backup uname fromdos todos
#@@ cp-rf crontab play listpkg installpkg removepkg
E=echo

for i in system.cfg live.cfg chat.pwd;do
[ -f /etc/$i ] && . /etc/$i
done
...
p=`basename $0`
case $p in
...
grep) E="$1"
case $1 in
-h) $E Usage: grep [-v] STRING [files ...];;
-v) E="$2"; shift 2; sed -n /$E/!P "$@";;
*) shift; sed -n /$E/P "$@";;
esac;;
...

You can see the basic grep part in blue. If you have installed bash (I hate the default ile command-line) you can play around:

[Linux] alias g=/pkg/bin/grep
[Linux] g a --version
grep (GNU grep) 2.3

Copyright (C) 1988, 1992-1998, 1999 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[Linux] echo $?
0

If the result of 'g' is zero, you have GNU grep available.

[Linux] PKG=""
[Linux] $PKG/bin/grep a --version
sed: can't open --version
[Linux] [ $? != 0 ] && echo "You are using sed-grep"
You are using sed-grep

In the example above, you can see and proove the default usage of grep in Freesco.

This is not a page of extensive commands. If you want to learn more, go at www.freesco.org/home/inc/commands.htm

One more service for Dynamic dial-up IPs

The best site I know to allow you to have a CNAME for your dynamic dial-up IP is DynDNS (www.dyndns.com). Especially, the basic service is free of any charge.
If you want another service, I have recently tried indeed:

No-IP (www.no-ip.com)

It supports a fairly easy client, named noip. The version I have tried is noip-2.1.4, and it worked smoothly after compilation.

./configure
make

A command is available to generate the configuration file (noip -C). Then I have launched it with maximum debug output:

[henrique@fuji noip-2.1.4]$ ./noip2 -c noip-test.conf -d -d -d -d -d

I have no idea about the algorithm behind, but it queried the no-ip server with a reasonably scrambled username / password. Something I have never seen with other services.

HTTP chunks

The self-made tool gnetinfo contains the basic mechanisms of a browser. I did need to understand most of the concepts explained below before doing actual implementation.
The output shown was sniffed in eth0 at my development server -- using gpackcount (part of netpack package).

An example of a chunked transaction
Page layout

<...>Apache/1.3.33 Server at fuji.prized Port 80

HTTP conversation

In grey-ink it is signalled the second connection made by the browser (located at a host named luisa, on our prized network). In light grey the unimportant TCP conversation. [The colors show correctly in the original doc, probably broken at this post.)

{TCP/IP} (size=232, str=292) luisa.prized:1124 => 0.0.0.0:80 GET /[7E]henrique/bump/ HTTP/1.1[0D][0A]Accept: */*[0D][0A]Referer: http://fuji/[7E]henrique/[0D][0A]Accept-Language: pt[0D][0A]Accept-Encoding: gzip, deflate[0D][0A]User-Agent: Mozilla/4.0 (compatible; MSIE x.y)[0D][0A]Host: fuji[0D][0A]Connection: Keep-Alive[0D][0A][0D][0A]

Apache/1.3.33 Server at fuji.prized Port 80{TCP/IP} (size=1118, str=1271) 0.0.0.0:80 => luisa.prized:1124 HTTP/1.1 200 OK[0D][0A]Date: Tue, 12 Jun 2007 18:34:55 GMT[0D][0A]Server: Apache/1.3.33 (Unix) PHP/4.3.10[0D][0A]Keep-Alive: timeout=15, max=100[0D][0A]Connection: Keep-Alive[0D][0A]Transfer-Encoding: chunked[0D][0A]Content-Type: text/html[0D][0A][0D][0A]383[0D][0A][0A][0A] [0A] [0A] [0A] [0A]

Index of /[7E]henrique/bump...[0A][0A][0A][0D][0A]0[0D][0A][0D][0A]

{TCP/IP} (size=323, str=392) luisa.prized:1124 => 0.0.0.0:80 GET /icons/blank.gif HTTP/1.1[0D][0A]Accept: */*[0D][0A]Referer: http://fuji/[7E]henrique/bump/[0D][0A]Accept-Language: pt[0D][0A]Accept-Encoding: gzip, deflate[0D][0A]If-Modified-Since: Thu, 22 Feb 1996 11:45:54 GMT[0D][0A]If-None-Match: "f3b5b-94-312c5772"[0D][0A]User-Agent: Mozilla/4.0 (compatible; MSIE x.y)[0D][0A]Host: fuji[0D][0A]Connection: Keep-Alive[0D][0A][0D][0A]

{TCP/IP} (size=0, str=0) luisa.prized:1125 => 0.0.0.0:80

{TCP/IP} (size=0, str=0) 0.0.0.0:80 => luisa.prized:1125

{TCP/IP} (size=6, str=24) luisa.prized:1125 => 0.0.0.0:80 [00][00][00][00][00][00]

{TCP/IP} (size=322, str=391) luisa.prized:1125 => 0.0.0.0:80 GET /icons/back.gif HTTP/1.1[0D][0A]Accept: */*[0D][0A]Referer: http://fuji/[7E]henrique/bump/[0D][0A]Accept-Language: pt[0D][0A]Accept-Encoding: gzip, deflate[0D][0A]If-Modified-Since: Thu, 22 Feb 1996 11:45:53 GMT[0D][0A]If-None-Match: "f3b51-d8-312c5771"[0D][0A]User-Agent: Mozilla/4.0 (compatible; MSIE x.y)[0D][0A]Host: fuji[0D][0A]Connection: Keep-Alive[0D][0A][0D][0A]

{TCP/IP} (size=202, str=244) 0.0.0.0:80 => luisa.prized:1124 HTTP/1.1 304 Not Modified[0D][0A]Date: Tue, 12 Jun 2007 18:34:55 GMT[0D][0A]Server: Apache/1.3.33 (Unix) PHP/4.3.10[0D][0A]Connection: Keep-Alive, Keep-Alive[0D][0A]Keep-Alive: timeout=15, max=98[0D][0A]ETag: "f3b5b-94-312c5772"[0D][0A][0D][0A]

{TCP/IP} (size=324, str=393) luisa.prized:1124 => 0.0.0.0:80 GET /icons/folder.gif HTTP/1.1[0D][0A]Accept: */*[0D][0A]Referer: http://fuji/[7E]henrique/bump/[0D][0A]Accept-Language: pt[0D][0A]Accept-Encoding: gzip, deflate[0D][0A]If-Modified-Since: Thu, 22 Feb 1996 11:46:04 GMT[0D][0A]If-None-Match: "f3b7a-e1-312c577c"[0D][0A]User-Agent: Mozilla/4.0 (compatible; MSIE x.y)[0D][0A]Host: fuji[0D][0A]Connection: Keep-Alive[0D][0A][0D][0A]

{TCP/IP} (size=202, str=244) 0.0.0.0:80 => luisa.prized:1125 HTTP/1.1 304 Not Modified[0D][0A]Date: Tue, 12 Jun 2007 18:34:55 GMT[0D][0A]Server: Apache/1.3.33 (Unix) PHP/4.3.10[0D][0A]Connection: Keep-Alive, Keep-Alive[0D][0A]Keep-Alive: timeout=15, max=99[0D][0A]ETag: "f3b51-d8-312c5771"[0D][0A][0D][0A]

{TCP/IP} (size=202, str=244) 0.0.0.0:80 => luisa.prized:1124 HTTP/1.1 304 Not Modified[0D][0A]Date: Tue, 12 Jun 2007 18:34:55 GMT[0D][0A]Server: Apache/1.3.33 (Unix) PHP/4.3.10[0D][0A]Connection: Keep-Alive, Keep-Alive[0D][0A]Keep-Alive: timeout=15, max=97[0D][0A]ETag: "f3b7a-e1-312c577c"[0D][0A][0D][0A]

{TCP/IP} (size=6, str=24) luisa.prized:1125 => 0.0.0.0:80 [00][00][00][00][00][00]

{TCP/IP} (size=6, str=24) luisa.prized:1124 => 0.0.0.0:80 [00][00][00][00][00][00]

Background about browsed files

One of the files referred was the ‘folder’ icon, shown below.

% ll --full-time /www/icons/back.gif

-rw-r--r-- 1 henrique user 216 Thu Feb 22 12:45:53 1996 [...]back.gif

Uninteresting fields within transaction

The following field (marked in yellow):

If-None-Match: "f3b51-d8-312c5771"

is, from my point of view, only interesting for the cache of the browser (http://www.freesoft.org/CIE/RFC/2068/187.htm), nothing more.

Interesting fields within transaction

The server responded:

“HTTP/1.1 304 Not Modified”

because the file (an image, actually “back.gif”) didn’t change since the last time the browser did fetch it.

[Old] discussion about HTTP chunks

I always wondered why chunking is necessary at all. Not only is necessary but is considered efficient. There are proofs over that statements referred below (highlighted in yellow).

Re: HTTP/1.1 : Chunking

John Franks (john at ath.nwu.edu)
Thu, 29 Jan 1998 11:53:35 -0600 (CST)

On Thu, 29 Jan 1998, Adrien de Croy wrote:

> 

> 1. The chunk size may be arbitrary, and the resulting latency introduced at

> the server (since chunking at the server requires some form of buffering)

> may cause performance degradation in applications such as streamed media

> through HTTP.  Is there any recommendation on how to chunk the data?

> 

The server may chunk anyway it chooses.  It can choose to do so in a

way it finds most efficient.

> 2. Any intermediary (proxy) must continually monitor the transfer through

> itself. Is it allowed to re-chunk the data, and if so under what guidelines.

> 

It is not only allowed, but expected to be the most common practice

for caching proxies.  The proxy can re-chunk anyway it chooses.

> 

> Proposal:

> 

> Addition of a new option to specify that transmission of the entity is

> terminated by a certain sequence of octets (like an end tag) transmitted by

> the server before closing the TCP connection.  The receiver would then know

> if had received the whole entity by examining the last packet received

> before the connection was closed.  If the end tag was there, then it got the

> lot, else it didn't - simple. 

> 

As you said, the point of chunking is so the receiver can know when

the entity ends.  But the point of knowing when the entity ends is not

only to recognize if the connection closed prematurely.  The main

point is to allow multiple HTTP transactions to occur in one TCP/IP

connection.  This is a big performance win.  This means, though, that

the entity does not end with the last packet and, in fact, entities

will typically end in the middle of a packet.  This makes it less

efficient (among other problems) to use an end marker.  The receiver

would have to read all the incoming data and look for the marker.

Chunking is both more robust and more efficient.

John Franks

john at math.nwu.edu

end of citation, quoted from http://www.hpl.hp.com/personal/ange/archives/http-wg-archive/0274.html