PerlMonks - The Monastery Gates

http://www.perlmonks.org/

WebOS

http://en.m.wikipedia.org/wiki/WebOS

Ozma: Extending Scala With Oz Concurrency

I want to go through this to understand it better.  The "Unusual Syntax" sounds like a major hurdle and most of these feartures probably could be added to more traditional languages.  C++, JAVA, RUBY, PERL, etc.



From Slashdot: Announcing Ozma: Extending Scala With Oz Concurrency

"Programming for concurrency makes sense in developing for both large scales (cloud computing) and small (multicore CPUs). Some languages were designed for concurrency and distribution; One of those languages is Oz, which provides advanced primitives regarding concurrency and distribution. Oz is mostly declarative, a paradigm that encompasses functional and logic programming. Despite its innovative features and expressiveness, Oz never made it into the wide developer community; one reason is its unusual syntax." Read on to learn about an effort to bring Oz's concurrency features to more programmers. setori88 continues: "But first, some background: Martin Odersky, in designing the Scala programming language, recognized the need for community acceptance of the kind that Oz lacked. He designed Scala as being both close to Java (in terms of syntax and concepts) and interoperable with existing Java libraries. Today, Scala seems to be the best hope for making functional programming accessible to programmers.

Although Scala has no language feature related to concurrency, the advanced library Akka, inspired by Erlang, provides Scala programmers with concurrent and distributed concepts.

Now comes a project attempting to popularize the concurrency concepts of Oz, called Ozma. Ozma implements the full Scala specification and runs on the Mozart VM. It extends Scala with dataflow variables, declarative concurrency, lazy declarative concurrency, and message-passing concurrency based on ports. Ozma extends the duality of Scala, namely the combination of functional and object styles, to concurrent programming."

The Mozart Programming System
http://www.mozart-oz.org/ The Mozart Programming System is an advanced development platform for intelligent, distributed applications. The system is the result of a decade of research in programming language design and implementation, constraint-based inference, distributed computing, and human-computer interfaces.

Mozart is based on the Oz language, which supports declarative programming, object-oriented programming, constraint programming, and concurrency as part of a coherent whole. For distribution, Mozart provides a true network transparent implementation with support for network awareness, openness, and fault tolerance. Mozart supports multi-core programming with its network transparent distribution and is an ideal platform for both general-purpose distributed applications as well as for hard problems requiring sophisticated optimization and inferencing abilities. We have developed many applications including sophisticated collaborative tools, multi-agent systems, and digital assistants, as well as applications in natural language understanding and knowledge representation, in scheduling and time-tabling, and in placement and configuration.

MemCacheD

http://en.wikipedia.org/wiki/Memcached

Manage Open Source Software | Black Duck Software

http://www.blackducksoftware.com/

Li Mei’s list of debugging secrets

http://www.eetimes.com/design/embedded/4217189/Li-Mei-s-list-of-debugging-secrets?cid=NL_Embedded&Ecosystem=embedded

BayLISA had a some great talks last week

BayLISA a group dedicated to Sysadmins, had a some great talks last week.

• Force 10 bare metal provisioning demo.  (IaaS, Cloud component)
• Arista Networks vmtracer demo
• Logstash, with Jordan Sissel 
• FreeNAS BSD based Open Source NAS sponsored by IX Systems.

Logstash really looks a lot like a Open source version of Splunk.

What is the Cloud?

Components of a Cloud.
There are three main categories of cloud computing service, called the SPI model (SaaS, PaaS, IaaS).

Anything as a Serivce (XaaS) is a term used to all of these.

1. Software as a Service (SaaS) is a software distribution model in which applications are hosted by a vendor or service provider and made available to customers over a network, typically the Internet.
2. Platform as a Service (PaaS) is a paradigm for delivering operating systems and associated services over the Internet without downloads or installation.
3. Infrastructure as a Service (IaaS) involves outsourcing the equipment used to support operations, including storage, hardware, servers and networking components.

What is Infrastructure as a Service (IaaS)?
Infrastructure as a Service is a provision model in which an organization outsources the equipment used to support operations, including storage, hardware, servers and networking components. The service provider owns the equipment and is responsible for housing, running and maintaining it. The client typically pays on a per-use basis.

Characteristics and components of IaaS include:

• Utility computing service and billing model.
• Automation of administrative tasks.
• Dynamic scaling.
• Desktop virtualization.
• Policy-based services.
• Internet connectivity.
• Virtual Servers, and Virtualization

Sometimes referred to as (HaaS) Hardware as a Service.


Platform as a Service (PaaS) is a way to rent hardware, operating systems, storage and network capacity over the Internet. The service delivery model allows the customer to rent virtualized servers and associated services for running existing applications or developing and testing new ones.

Platform as a Service (PaaS) is an outgrowth of Software as a Service (SaaS), a software distribution model in which hosted software applications are made available to customers over the Internet. PaaS has several advantages for developers. With PaaS, operating system features can be changed and upgraded frequently. Geographically distributed development teams can work together on software development projects. Services can be obtained from diverse sources that cross international boundaries. Initial and ongoing costs can be reduced by the use of infrastructure services from a single vendor rather than maintaining multiple hardware facilities that often perform duplicate functions or suffer from incompatibility problems. Overall expenses can also be minimized by unification of programming development efforts.
On the downside, PaaS involves some risk of "lock-in" if offerings require proprietary service interfaces or development languages. Another potential pitfall is that the flexibility of offerings may not meet the needs of some users whose requirements rapidly evolve.
 

Software as a Service (SaaS) is a software distribution model in which applications are hosted by a vendor or service provider and made available to customers over a network, typically the Internet.

SaaS is becoming an increasingly prevalent delivery model as underlying technologies that support Web services and service-oriented architecture (SOA) mature and new developmental approaches, such as Ajax, become popular. Meanwhile, broadband service has become increasingly available to support user access from more areas around the world.
SaaS is closely related to the ASP (application service provider) and on demand computing software delivery models. IDC identifies two slightly different delivery models for SaaS. The hosted application management (hosted AM) model is similar to ASP: a provider hosts commercially available software for customers and delivers it over the Web. In the software on demand model, the provider gives customers network-based access to a single copy of an application created specifically for SaaS distribution.
Benefits of the SaaS model include:

• easier administration
• automatic updates and patch management
• compatibility: All users will have the same version of software.
• easier collaboration, for the same reason
• global accessibility.

The traditional model of software distribution, in which software is purchased for and installed on personal computers, is sometimes referred to as software as a product.


SaaS can be build on PaaS on IaaS


-------------------------------
This is still a draft article from here on down.
I am in the process of still writting.
John 6/17/2011
-------------------------------

Examples:

• Amazon web services, BeanStalk, Elastic Compute EC2
• VMware
• Windows Azure
• SalesForce.com
• Google Gmail
• CloudBees

Scale out vs. Scale up.

Tools:
Enterprise Java three tier application
MySQL, MongoDB
NoSQL, CouchDB

Memory or Cache (Redis, memcached, Gemfire)

RabbitMQ, AMQP, JMS for messaging in the cloud. 
Amazon SNS, SQS

POJO? Apache Hadoop?  HDFS, HBase, Hive

Makara, an open-source software-project for cloud deployment and management managed by Red Hat

MAP/Reduce

CAP Theorem
C: Consistency: All nodes see the same data
A: Availablity: Node failure does not prevent survivors from operating
P: Partition Tolerance: System works even though message are lost

http://en.wikipedia.org/wiki/Cloud_computing
http://en.wikipedia.org/wiki/List_of_cloud_computing_providers
http://en.wikipedia.org/wiki/Cloud_%28operating_system%29

Redis key-value store server.

Redis is an advanced key-value store.

It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets.

Written in ANSI C, it's open-source, networked, in-memory, persistent and  journaled. 

Supported languages or language bindings include C, C++, C#, Clojure, Common Lisp, Erlang, Haskell, Java, JavaScript, Lua, Objective-C, Perl, PHP, Python, R, Ruby, Scala, Go, and Tcl.
As of 15 March 2010, development of Redis is sponsored by VMware.

http://redis.io/
http://en.wikipedia.org/wiki/Redis (data store)
Redis: Lightweight key/value Store That Goes the Extra Mile
Evolving the Key/Value Programming Model to a Higher Level 

Advanced Message Queuing Protocol

The Advanced Message Queuing Protocol (AMQP) is an open standard application layer protocol for message-oriented middleware (MOM).  The defining features of AMQP are message orientation, queuing, routing (including point-to-point and publish-and-subscribe), reliability and security.


It's similar to JMS for Java but not language specific. 

http://www.amqp.org/confluence/display/AMQP/Advanced+Message+Queuing+Protocol

Linux based PID Servomotor Control

Great article that talk about implemented the PID, or the Proportional Integral Derivative, algorithm to make a 2 wheeled robot that balences like a segway. 

Linux based PID Control

How Skype gets round firewalls

UPDATE: This protocol is now incorporated in to STUN , TURN and ICE.     Global IP Solutions, the company that made the VOIP system that Skype was using was bought by Google shortly after Skype developed their own technology.  Google then made the original skype technology freely available as WebRTC now supported by many HTML5 browsers.  Microsoft then bought Skype. ----- REPOST FROM: http://www.heise-security.co.uk/articles/82481  No longer on the web. I want to point out this is the same method described in my Paper: Method of passing bi-directional data between two firewalls. 12 /15/2006

 

Jürgen Schmidt

The hole trick

How Skype & Co. get round firewalls

Peer-to-peer software applications are a network administrator's nightmare. In order to be able to exchange packets with their counterpart as directly as possible they use subtle tricks to punch holes in firewalls, which shouldn't actually be letting in packets from the outside world.
Increasingly, computers are positioned behind firewalls to protect systems from internet threats. Ideally, the firewall function will be performed by a router, which also translates the PC's local network address to the public IP address (Network Address Translation, or NAT). This means an attacker cannot directly address the PC from the outside - connections have to be established from the inside.
This is of course a problem when two computers behind NAT firewalls require to talk directly to each other - if, for example, their users want to call each other using Voice over IP (VoIP). The dilemma is clear - whichever party calls the other, the recipient's firewall will decline the apparent attack and will simply discard the data packets. The telephone call doesn't happen. Or at least that's what a network administrator would expect.

Punched

But anyone who has used the popular internet telephony software Skype knows that it works as smoothly behind a NAT firewall as it does if the PC is connected directly to the internet. The reason for this is that the inventors of Skype and similar software have come up with a solution.
Naturally every firewall must also let packets through into the local network - after all the user wants to view websites, read e-mails, etc. The firewall must therefore forward the relevant data packets from outside, to the workstation computer on the LAN. However it only does so, when it is convinced that a packet represents the response to an outgoing data packet. A NAT router therefore keeps tables of which internal computer has communicated with which external computer and which ports the two have used.
The trick used by VoIP software consists of persuading the firewall that a connection has been established, to which it should allocate subsequent incoming data packets. The fact that audio data for VoIP is sent using the connectionless UDP protocol acts to Skype's advantage. In contrast to TCP, which includes additional connection information in each packet, with UDP, a firewall sees only the addresses and ports of the source and destination systems. If, for an incoming UDP packet, these match an NAT table entry, it will pass the packet on to an internal computer with a clear conscience.

Switching

The switching server, with which both ends of a call are in constant contact, plays an important role when establishing a connection using Skype. This occurs via a TCP connection, which the clients themselves establish. The Skype server therefore always knows under what address a Skype user is currently available on the internet. Where possible the actual telephone connections do not run via the Skype server; rather, the clients exchange data directly.
Let's assume that Alice wants to call her friend Bob. Her Skype client tells the Skype server that she wants to do so. The Skype server already knows a bit about Alice. From the incoming query it sees that Alice is currently registered at the IP address 1.1.1.1 and a quick test reveals that her audio data always comes from UDP port 1414. The Skype server passes this information on to Bob's Skype client, which, according to its database, is currently registered at the IP address 2.2.2.2 and which, by preference uses UDP port 2828.

Step 1: Alice tries to call Bob, which signals Skype.

Bob's Skype program then punches a hole in its own network firewall: It sends a UDP packet to 1.1.1.1 port 1414. This is discarded by Alice's firewall, but Bob's firewall doesn't know that. It now thinks that anything which comes from 1.1.1.1 port 1414 and is addressed to Bob's IP address 2.2.2.2 and port 2828 is legitimate - it must be the response to the query which has just been sent.

Step 2: Bob tries to reach Alice, which punches a hole through Bob's Firewall.

Now the Skype server passes Bob's coordinates on to Alice, whose Skype application attempts to contact Bob at 2.2.2.2:2828. Bob's firewall sees the recognized sender address and passes the apparent response on to Bob's PC - and his Skype phone rings.

Step 3: Alice finally reaches Bobs computer through the hole.

Doing the rounds

This description is of course somewhat simplified - the details depend on the specific properties of the firewalls used. But it corresponds in principle to our observations of the process of establishing a connection between two Skype clients, each of which was behind a Linux firewall. The firewalls were configured with NAT for a LAN and permitted outgoing UDP traffic.
Linux' NAT functions have the VoIP friendly property of, at least initially, not changing the ports of outgoing packets. The NAT router merely replaces the private, local IP address with its own address - the UDP source port selected by Skype is retained. Only when multiple clients on the local network use the same source port does the NAT router stick its oar in and reset the port to a previously unused value. This is because each set of two IP addresses and ports must be able to be unambiguously assigned to a connection between two computers at all times. The router will subsequently have to reconstruct the internal IP address of the original sender from the response packet's destination port.
Other NAT routers will try to assign ports in a specific range, for example ports from 30,000 onwards, and translate UDP port 1414, if possible, to 31414. This is, of course, no problem for Skype - the procedure described above continues to work in a similar manner without limitations.
It becomes a little more complicated if a firewall simply assigns ports in sequence, like Check Point's FireWall-1: the first connection is assigned 30001, the next 30002, etc. The Skype server knows that Bob is talking to it from port 31234, but the connection to Alice will run via a different port. But even here Skype is able to outwit the firewall. It simply runs through the ports above 31234 in sequence, hoping at some point to stumble on the right one. But if this doesn't work first go, Skype doesn't give up. Bob's Skype opens a new connection to the Skype server, the source port of which is then used for a further sequence of probes.

Skype can do port scans. Here it succeeds on port 38901 and connects through the firewall.

Nevertheless, in very active networks Alice may not find the correct, open port. The same also applies for a particular type of firewall, which assigns every new connection to a random source port. The Skype server is then unable to tell Alice where to look for a suitable hole in Bob's firewall.
However, even then, Skype doesn't give up. In such cases a Skype server is then used as a relay. It accepts incoming connections from both Alice and Bob and relays the packets onwards. This solution is always possible, as long as the firewall permits outgoing UDP traffic. It involves, however, an additional load on the infrastructure, because all audio data has to run through Skype's servers. The extended packet transmission times can also result in an unpleasant delay.
Use of the procedure described above is not limited to Skype and is known as "UDP hole punching". Other network services such as the Hamachi gaming VPN application, which relies on peer-to-peer communication between computers behind firewalls, use similar procedures. A more developed form has even made it to the rank of a standard - RFC 3489 "Simple Traversal of UDP through NAT" (STUN) describes a protocol which with two STUN clients can get around the restrictions of NAT with the help of a STUN server in many cases. The draft Traversal Using Relay NAT (TURN) protocol describes a possible standard for relay servers.

DIY hole punching

With a few small utilities, you can try out UDP hole punching for yourself. The tools required, hping2 and netcat, can be found in most Linux distributions. Local is a computer behind a Linux firewall (local-fw) with a stateful firewall which only permits outgoing (UDP) connections. For simplicity, in our test the test computer remote was connected directly to the internet with no firewall.
Firstly start a UDP listener on UDP port 14141 on the local/1 console behind the firewall:

local/1# nc -u -l -p 14141

An external computer "remote" then attempts to contact it.

remote# echo "hello" | nc -p 53 -u local-fw 14141

However, as expected nothing is received on local/1 and, thanks to the firewall, nothing is returned to remote. Now on a second console, local/2, hping2, our universal tool for generating IP packets, punches a hole in the firewall:

local/2# hping2 -c 1 -2 -s 14141 -p 53 remote

As long as remote is behaving itself, it will send back a "port unreachable" response via ICMP - however this is of no consequence. On the second attempt

remote# echo "hello" | nc -p 53 -u local-fw 14141

the netcat listener on console local/1 then coughs up a "hello" - the UDP packet from outside has passed through the firewall and arrived at the computer behind it.
Network administrators who do not appreciate this sort of hole in their firewall and are worried about abuse, are left with only one option - they have to block outgoing UDP traffic, or limit it to essential individual cases. UDP is not required for normal internet communication anyway - the web, e-mail and suchlike all use TCP. Streaming protocols may, however, encounter problems, as they often use UDP because of the reduced overhead.
Astonishingly, hole punching also works with TCP. After an outgoing SYN packet the firewall / NAT router will forward incoming packets with suitable IP addresses and ports to the LAN even if they fail to confirm, or confirm the wrong sequence number (ACK). Linux firewalls at least, clearly fail to evaluate this information consistently. Establishing a TCP connection in this way is, however, not quite so simple, because Alice does not have the sequence number sent in Bob's first packet. The packet containing this information was discarded by her firewall.

WebRTC - bringing real time communications to the web.

WebRTC - an open real-time communications project

WebRTC offers web application developers the ability to write rich, realtime multimedia applications (think video chat) on the web, without requiring plugins, downloads or installs. It's purpose is to help build a strong RTC platform that works across multiple web browsers, across multiple platforms.


https://sites.google.com/site/webrtc/blog

Low Level Virtual Machine (LLVM)

LLVM is a really interesting alternative to java and it's bytecode jvm.
What's interesting is it can be run in the web browsers Javascript enterpreter allow this to run on web site.

So you can compile C & C++ code to run on the web in a javascript VM.

> Deepak  - Apple is funding the project.llvm-gcc is the default compiler which is shipped with Mac OS and IOS SDK.


http://llvm.org/

Low Level Virtual Machine (LLVM) is a compiler infrastructure, written in C++, which is designed for compile-time, link-time, run-time, and "idle-time" optimization of programs written in arbitrary programming languages. Originally implemented for C/C++, the language-independent design (and the success) of LLVM has since spawned a wide variety of front ends, including Objective-C, Fortran, Ada, Haskell, Java bytecode, Python, Ruby, ActionScript, GLSL, and others.

Emscripten is an LLVM-to-JavaScript compiler. It takes LLVM bitcode (which can be generated from C/C++, using llvm-gcc or clang, or any other language that can be converted into LLVM) and compiles that into JavaScript, which can be run on the web (or anywhere else JavaScript can run).

From Slashdot: Doom Ported To the Web

"Ever since Id Software released the Doom source code under the GPL, it's been ported to platform after platform. Now, you can play Doom compiled to JavaScript on the web, using standard web technologies like Canvas and without any plugins. If your browser has trouble running it, here's a screencast." The translation was accomplished using Emscripten, a Javascript backend for LLVM. As per the GPL, full source code is available. Pretty neat.