Tech

Legendary writer and author Jerry Pournelle sent me an email! OK, so I’m a subscriber on his website and he sent it out to everybody, but still, I thought it was cool. The message was a warning about the fact that malicious hackers had compromised the online job boards and were selling their services to spammers and scam artists. Monster.com, hotjobs.com, and other mainstream job boards are affected. You can get the details here.

I became a fan of Jerry’s writing in the ’80s when he was a columnist for BYTE Magazine. This was back when magazines printed a lot of useful technical information, hacks, program listings, and electronics projects. A fellow programmer was a subscriber, and introduced me to the magazine, and particularly “Chaos Manor,” Jerry’s column. When BYTE was sold to another publisher, it’s format turned more to product reviews and coverage of the IT business industry. In other words, it became geared more for managers with IT budgets and less for programmers, hobbyists, and end users. Not long after that the magazine ceased publication altogether.

I missed Jerry’s anecdotal accounts of his struggles with technology. He did product reviews too, but always from the first-person perspective of a non-technical person (an author) actually trying to implement, rather than just cover, the products. His often humorous tales were always informative and entertaining. I was delighted when I discovered a year or so ago that Jerry was still writing his Chaos Manor Reviews, as well as Other Musings. I should have known that his talent and desire (need?) for self-expression would have steered him toward the online publishing world, and was chagrined that I hadn’t thought to search on his name sooner.

My home server, which acts as a development web server as well as a file and print server for the numerous PCs around my household, was running on SUSE Linux 10.0. This version of SUSE was the only one of its kind, before Novel re-branded the free version of the OS as OpenSUSE. It was getting increasingly difficult for me to get the updates for SUSE Linux 10.0, so I figured it was time to upgrade to something a little newer. Of course I was planning on loading OpenSUSE 10.3, the current stable release.

Looking for ultra-light linux for kids’ PC

About this time my kids’ old Window 98 PC seemed to be grinding down to a halt as they tried to get it to do more - IM, online games, etc. I tried a couple of small linux loads known for good performance on old hardware, like Puppy Linux and Damn Small Linux (DSL). While both of these are great distros, on this hardware Puppy Linux bogged down with too little memory, and DSL had problems with the USB wireless network adapter on that PC. I even put in an old 16 bit Intel NIC and spent a couple of evenings fishing CAT-5 cable through the attic and wall, but still had some problems with DSL recognizing it.

Someone Mentioned FreeBSD

On some forum (can’t recall where), someone mentioned FreeBSD as a viable option for old, slow hardware, so I decided to try it out on the kids’ PC. After some tweaking of the device hints to get it to use the old NIC, it worked great. Performance is acceptable for the hardware (300 mhz Pentium II), and it runs most Linux software in addition to native FreeBSD code. After some more online research about the relative strengths and weaknesses of FreeBSD vs. other BSDs vs. Linux, I decided to reload the server with FreeBSD. This server is built on a DELL Dimensions platform, with a 2 Gigahertz Intel processor and 256 Meg of memory. It has 2 internal hard drives, a 30 Gig Maxtor 6E030L0, and a 160 Gig Seagate ST3160215A (which came out of a failed external USB SimTech drive.

The Plan

Since the Seagate had a lot of unused space, I wanted to use part of it to back up stuff I wanted to save, but I also knew I would need some of it free for the FreeBSD installation. The trouble is, it was formatted as a ReiserFS file system, which FreeBSD can read but not write to. So the plan included steps to shrink that file system and leave enough unallocated space for FreeBSD to create usable storage. So the steps in general were:

1. Estimate space required to backup user data and development work.
2. Shrink the ReiserFS file system to create unallocated space on the disk, but leave enough on ReiserFS to accommodate the backup.
3. Change the partition sizes on the disk (shrinking the file system does not affect the underlying partition size).
4. Backup desired files.
5. Download FreeBSD and create installation CDs.
6. Install
7. Restore files and configure applications (samba, cups, apache, quanta Plus, etc.)

Next: Part 1 (coming soon): Using resize_reiserfs and cfdisk.

Several months ago I wrote an article for a Helium Marketplace publisher.  My submission was not selected, but I find that it is one of my more popular articles on Helium (definitely in the top 3), so I thought I’d share it here.  Please read How to find out what’s running on your PC (and why this is important).  Let me know how you like it.

The other night I spent about 2 1/2 hours at the house of a friend, trying to free his PC of some particularly nasty malware, SpyGuarder and Vista AntiVirus 2008. Both are classified as rogue anti-spyware programs. This type of malware attempts to trick you into buying their full versions by running free scans with a trial version, and showing you all sorts of viruses, trojans, keyloggers, etc. with which your system is supposedly infected. They then offer to remove all these infections if you’ll just click the link and upgrade to the full version of their program, which of course, costs money. There are a number of problems with both these programs.

1. Your system doesn’t really have the infections these programs claim. Or to be more accurate, they have no way of knowing one way or another, since the so-called “scans” they do are completely fake. Nor could they remove the infections if you did have them, since they do not actually fight spyware or viruses, but are likely to install some of their own. Of course, if you elect to do this, future scans will say that your system is now clean.
2. These programs are obnoxiously persistent. Any attempts to cancel the scans, close the windows, or kill the processes just result in another process being launched.
3. These programs prevent legitimate anti-spyware programs from installing and running. Generally, when trying to clean spyware out of a system, one of the first things I do is install and run Adaware from lavasoft. Vista AntiVirus 2008 would not let me install it, popping up a fake system message saying basically that the Administrator for the PC has configured it to disallow “installations of this type.” Spybot Search & Destroy did work, but did not remove the two nasties I was dealing with. SpyGuarder similarly prevents the task manager from launching, claiming that “Task Manager has been disbled by your Administrator.”
4. The presence of either of these programs indicates that you may have the zlob or other dangerous trojans.

No doubt some of you would have advised me to run various legitimate anti-malware applications like SpyHunter, which can apparently automate the removal of SpyGuarder and Vista AntiVirus 2008. Pride and miserliness made me opt to do it by hand, which I did with the help of instructions found here and here.

Vista AntiVirus 2008 has several other identities, all which do the same bad things to your system, such as Windows Antivirus 2008, Windows AntiVirus Pro, etc. These, as well as SpyGuarder, are advertised on professional-looking web sites, and give the appearance of being the most advanced anti-malware products on the market. Do not be fooled, do not install either of these products - the commercial or the free versions - on your computer under any circumstances. If you find you have been infected with either of these anyway (it’s possible to pick them up via “drive-by” infection), take steps to remove them immediately.

Added 7/1/2008: I had to go back and remove yet another fake security program. His commercial virus protection had long since expired, so I installed AVG Free, which found and removed about a dozen viruses and trojans, but then his desktop and taskbar disappeared. After searching around on the internet, I found that Malwarebyte’s RogueRemover Free is a great free tool which completely fixed the desktop problems and removed some additional adware / spyware. It will definitely be joining Adaware and Spybot Search & Destroy in my arsenal.

Anyone who’s used a personal computer for more than a week or two has undoubtedly noticed a gradual decrease in performance. There can be a number of causes for this, and a number of steps you can take to recover this lost performance. There are also a few preventative measures that can help keep your computer running at top efficiency.

Spyware and Adware - sources and removal
Spyware and Adware are two types of malicious software (AKA malware) that infect PCs. Spyware collects information about a user’s surfing habits, purchasing preferences, etc. and sends it to marketing agencies. Adware presents unwanted advertisements to the user. The source of infection can be email attachments or files downloaded from the internet disguised as or embedded within useful software. Some adware and spyware can also be picked up simply by surfing to certain websites.
Removal is usually accomplished with the aid of utilities written for this purpose. Spybot Search & Destroy and Adaware are two long-standing products which offer free versions for personal, home use. Some objects embed themselves so deeply within the operating system that free tools cannot completely remove them. For those, or if you’re running in a corporate environment and want continuous updates and real-time protection, consider a commercial offering.

Unnecessary Services and Processes
The default installation of Windows(c) configures a number of services that run automatically whenever the system is booted, many of which are never needed by the majority of users. Stopping these processes and preventing them from running can free up significant memory and CPU utilization. There are utilities that can make the job of identifying and disabling unnecessary processes easier. Some of this can be accomplished using Windows’ services interface. Getting to this interface differs between versions of Windows, but it will be similar to this: Start->Control Panel->Administrative Tools->Services. Here you will see the list of installed services. For each one you don’t want to run automatically every time you boot up, right-click on the name and select ‘Properties.’ In the dialog box, set the Startup Type to ‘manual.’ If you’re sure you never want the service to run (for example, if you suspect it is some kind of malware), set it to ‘disabled.’ You can always change it back to ‘automatic’ if you experience problems. Once you’ve finished setting the startup type on any services you’ve modified, you should reboot your computer. Simply stopping a service does not always completely free up resources that may have been reserved. The following are some services that are rarely needed by most users: Messenger Service (has nothing to do with instant messenger (IM) software), Remote Registry Service (do you ever need to edit your registry from a remote location?), Error Reporting Service (pops up the annoying “notify Microsoft about this bug” every time something crashes), Alerter (no need for this), Fast User Switching Compatibility (even with this disabled, you can still log off and log back on as someone else), Telnet (if you must enable a command-line log on from a remote location, use a secure shell (SSH) service instead).
There are other services which you may be able to disable, and there are other (non-service) processes that may be started by Windows. You can see which processes are running on your system by running the task manager (Ctrl-Alt-Del -> Task Manager) and selecting the Processes tab. These are started from registry entries, items in the Startup folder, and a number of other sources. With the task manager you can kill any of these processes (if you know which ones should be killed), but unless you find out where they’re coming from and remove the source, they will automatically restart. All these processes (including the services) can be managed with a program called Wintask 5 (liutilities.com). This tool gives you access to one of the most complete process libraries available, with the ability to identify, remove, or block undesirable processes. It costs about $30.00, but a free trial can be downloaded from the company’s website.

Optimize the Hard Drive(s)
Most people realize that they have to defragment their Hard Drives periodically or disk performance will suffer. Windows’ built-in defragmenter does an adequate job of defragmenting most files, but it has it’s limitations. Certain system files (including the registry) won’t be defragmented. Also, with this utility running, you can’t use your system for anything else. This program is actually a ‘light’ version of Diskeeper (diskeeper.com), which also comes in commercial flavors starting at about $30.00. For that price you get more efficient and complete defragmentation which can work in real-time, utilizing unused CPU cycles (so it doesn’t slow your system down).

Clean the Registry
The last thing I generally do when optimizing a system is to clean / optimize the registry. This removes references to obsolete objects and redundant entries, and repairs broken links. Again, this is accomplished with a utility. Remember to back up the registry first by using the File->Export menu option in the registry editor (regedit). The best type of utility for cleaning the registry is one that can defragment as well as clean it, something like RegistryBooster 2 (liutilities.com).
All of these optimization steps either require a utility or can be made easier with one. You can acquire free utilities or commercial variants. If you’re going to consider commercial software, you can save money by buying a suite. You can usually pick up a package deal for significantly less than the cost of individual components. Some, like PowerSuite from UniBlue, will also analyze and set the optimum parameters for your systems memory, CPU, and network configurations. Note that Powersuite includes a task manager, spyware removal and protection, and the RegistryBooster 2 registry cleaner, but alas, it does not include disk optimization.

Two of the worst causes of problems in personal computers these days is the prevalence of spyware and adware. Spyware and adware are types of malicious software (AKA malware) that infect PCs. Unlike other malware like viruses and trojans, spyware and adware don’t exist to cause damage directly, but to collect information about a user’s surfing habits, purchasing preferences, etc. and send it to marketing agencies (spyware), or to serve advertisements to the user, often making them appear as if they are normal pop-ups encountered while surfing the web (adware). Both these types of malware consume CPU cycles, memory, and network bandwidth, causing degradation in system performance and stability. Severe infections can make surfing the internet impossible or even render the entire system unusable. On top of that, spyware serves as an invasion of privacy, because the data collected can be used not only to target you with unwanted advertising, but quite possibly with identity theft as well.

The source of these infections can be email attachments, or files downloaded from the internet disguised as (or embedded within) useful software. Some adware and spyware can also be picked up simply by surfing to certain websites.

Removal of this type of malware is usually accomplished with the aid of utilities written for this purpose. There are free and commercially products available, each with their own set of strengths. Spybot Search & Destroy (safer-networking.org) and Adaware (lavasoft.de) are two long-standing products which offer free versions for personal, home use. Running scans with both these products, one after the other, will allow you to effectively remove most malware. Some objects however embed themselves so deeply within the operating system that free tools cannot completely remove them. There are a number of other tools available for dealing with these nefarious objects, each customized for the particular type of infection they’re designed to combat. For example, CWShredder (us.trendmicro.com) was designed to remove a rather insidious form of web browser hijacker, which redirects your searches, changes your home page, and creates bookmarks to other sites. Another tool for combatting hijackers and other malware is hijackthis, also from TrendMicro. Both of these tools are for experienced, technical users. You have to know specifically what you’re looking for. This is especially true of hijackthis, which will happily let you remove components that are actually quite critical to your system.

For these infections that are harder to find and kill, or if you’re running in a corporate environment and want continuous updates and real-time protection, you should consider a commercial offering. Adaware Pro sells for $39.00. The cost of the corporate edition of Sypot S & D is not given on their website.

An ounce of Prevention
A strictly commercial product (with a free trial) is >SpyEraser 2 from Uniblue ($29.95, uniblue.com). In addition to the ability to remove most spyware and adware, it offers real-time, continuous protection against becoming infected in the first place, and automatic daily updates. A free scan of your system is available from their website, as is an award-winning process library that can help you identify potentially dangerous processes that are running invisibly on your system.

Whether you decide to collect a set of free utilities or take the plunge and purchase a product depends on your level of expertise, the amount of free time you have to investigate and learn to use the various tools, and if you want or need the technical support that comes with a commercial product. In any case, it should be clear that you have to do something to combat spyware and adware on a regular basis if you want to keep your system running efficiently.

As reported a few months ago, I wrote a series of articles for an anonymous Helium Marketplace publisher related to PC Optimization. Well, they bought one - an article written ‘on spec’ about the benefits of a paid-for registry optimizer. Since I had an inkling that the publisher is Uniblue software, I made sure to mention their product. I did not mention it in the article, but my choice for free registry optimizers is CCleaner. Aside from that omission, the article has valid information concerning features to look for in a registry cleaner. An excerpt follows. For a limited time, the article can be read in it’s entirety. Once Uniblue publishes it, it will be removed from Helium, since they bought exclusive rights to the content.

---

This content was removed per the purchase agreement.  The original article can be read here, with someone else’s byline.  They can do that because they purchased exclusive rights. –Jp

This article was originally published by Triond on their web site ComputerSight. I thought it was time to reprint it here, so it appears below in its entirety.

---

Configuration Management (’CM’ hereafter) means a lot of different things to different people. Weighty tomes have been written describing the goals, policies, procedures, benefits, pitfalls, and a variety of definitions of CM. One recent CM plan I worked on is a 20-something page document attempting to detail this information and how it relates to the client’s projects.Most of the information available can be boiled down into 4 key concepts, or what can be called the 4 cornerstones of great CM. These concepts represent ideals. The challenge is in the implementation, so that the policies, procedures, and utilities developed support these ideals, or at least the intent behind them.

1. Version Control : Everything is maintained in a Version Control tool like Serena’s. Some agreed set of items (Configuration Items, or CI’s for short) stored within the tool represent baselines. In other words, they are the set of revisions currently in production. They are not necessarily the most recent revisions.Builds intended for deployment to any post-development environment (QA, Test, Prod, whatever) are always pulled from Version Control, and never copied directly from a development environment.
2. Separation of Duties and Least Privilege : Actually, these are two principles lumped together because Least Privilege is not possible without Separation of Duties, and Separation of Duties is pointless without Least Privilege. The former simply means that no single person has independent responsibility over more than one area of a system.For example, developers change code, perform unit test, etc., but do not deploy or promote such code to any non-development environment. CM people promote code, but do not develop applications, nor do they approve code changes made by developers (although they may participate in code reviews).
   DBAs have database privileges, but don’t develop application code nor act as system admins. And so on. The Least Privilege principle simply states that no person or running process has more access or system privilege than they need to perform their normal duties or functions at any point in time.Access or privilege for either people or processes can temporarily be increased during the performance of some activity as necessary, then immediately restricted again. Policies implementing these controls make allowances in both these principles for emergency situations.
3. Auditing : CM personnel periodically conduct audits of applications, systems, and procedures. Any updated application software or configurable item should be traceable to an approved change request, as well as through the entire set of existing quality control, tech review, and change control procedures.This includes not only application executables but database configurations as well. All items are compared with their baseline counterparts in the Version Control repository (ie; the revisions marked as ‘Production’). Discrepancies are reported as non-compliance issues and investigated, and will generally lead to procedural changes designed to eliminate future non-compliance.
4. Automate, Automate, Automate : This one is an over-riding theme for how we accomplish all this with limited resources. Checking items out of and into Version Control should be quick and painless, and integrated into development IDEs (Interactive Development Environments) if possible. Code promotions are scripted. Database changes are scripted. Auditing utilities are scripted.These scripts themselves are subject to review and kept in version control. Tying it all together gives us reliable, secure systems built with verifiable, repeatable and efficient processes.

The RIAA’s Investigators Operating Illegally?

The Recording Industry of America Association (RIAA) is the agency trying to enforce copyrights by suing suspected file sharers. They employ MediaSentry for the forensic examination of the computers owned by suspected violators, which opens up for scrutiny not only shared music, but any personal information stored on the PCs. Recently, MediaSentry has come under fire for conducting such examinations for evidence to be used in court cases without having Private Investigator (PI) licenses. Massachusetts has ordered them to cease operations there until they obtain the proper licenses. Several other states have issued various statements and warnings, including Michigan. This gives defendants the possibility of getting evidence disallowed in their trials, and opens the doors for recriminations from past defendants who settled out of court based on evidence that may not have been legally obtained. So far, MediaSentry has taken the stance that their role in the investigations does not require a PI license. This story bears watching closely. If anyone challenges the evidence gathered by MediaSentry, and it follows that they have to obtain PI licenses going forward, how many counter-suits from past cases will suddenly be filed by defendants who settled out of court on the strength of evidence that would not have been admitted in court?

Courts Can’t Force You To Reveal Your Passwords and Encryption Keys

Another case making headlines lately involves a Grand Jury’s attempt to order a defendant to reveal his encryption password so that prosecutors could assess the files on his hard drive. It turns out that passwords and encryption keys are protected under the 5th amendment, which basically prohibits the forcing of defendants to testify against themselves. A federal Magistrate ruled the Grand Jury’s subpoena unconstitutional. The government has appealed. If the suspect - a Canadian with U.S. residency by the name of Sebastien Boucher - is actually guilty of child pornography, I hope the government finds enough evidence to convict him without violating his 5th amendment rights.References:

• www.law.com reported on passwords and the 5th amendment.
• recordingindustryvspeople.blogspot.com brought the MediaSentry story to light.
• bloggernews.net - discusses the implications of the RIAA’s investigators operating illegally in certain states.

---

I first wrote this article as a submission to Helium’s Marketplace. It was not chosen, so my loss is your gain, as I’m posting it here in it’s entirety. Note that I received no compensation for this article, but since I was hoping WinRAR would choose to publish it, I can’t say it is completely unbiased. That being said, every point in the article is absolutely true, and the benchmarks referenced are available for anyone to view.

*** Original Article Below ***

Two of the most popular file compression programs on the market today are WinZIP and WinRAR. The question that inevitably arises is, which is a better buy? The answer can be found by comparing the performance, price, and features of each.

Compression - the whole purpose behind using compression software is to shrink the size of files for more efficient storage and faster transmission over a network. Thus, how well a utility compresses files should be of primary concern to those needing to choose one. Many independent sites on the Internet consistently show that WinRAR compresses files more than WinZIP. There are a few exceptions with specific file types, particularly those that are already compressed in their native format, like mp3 files. These do not compress very much in any case by either WinRAR or WinZIP. WinRAR comes out the clear winner among most other, highly compressible file types, and the aggregate or overall measures generally show WinRAR on top. Supporting data can be found on wikipedia (Comparison_of_file_archivers), techarp.com (Compression Comparison Guide Rev. 2.0), and maximumcompression.com. [click to continue...]

About 18 months ago I bought a SimpleTech 160 Gigabyte external USB hard drive for storing media files and backing up other data. I reformatted the drive so it would be writable from linux (using the reiserfs file system type). It worked great until about a month ago. At that time, whenever I rebooted, I had to power the drive off and on several times before linux would recognize it. Finally it stopped working altogether.

I tried moving it to another computer, but it wasn’t recognized there, either. The warranty on these drives is only 12 months, so I had no recourse with the company. With little to lose, I took the drive out of the enclosure to see what was under the cover. It was a Seagate Barracuda 160 Gigabyte Ultra IDE internal drive, with some circuitry to convert USB signals to IDE.

I removed the drive from the enclosure and installed it directly into the linux file server. Once I configured the mount point, it now works like a charm. I gave up the convenience of moving the drive from machine to machine, but I gained quite a bit of performance, and - more importantly - retained the data that had accumulated on the drive.

I have since read in various forums that this type of failure of external drives is fairly common. Most people fix it by buying a generic external enclosure to replace the original one. If you really need the portability, this is a better way to go. If you’re like me however, who just bought the external drive for ease of installation and because it was on sale, removing the drive from it’s enclosure and mounting it inside your computer might be a good way to recover a failing device.

One final note: I find it interesting that SimpleTech only warrants the product for 12 months, but according to Seagate’s web site, the Barracuda carries a 5 year warranty. It’s almost as if SimpleTech is admitting that the most likely point of failure is in the enclosure they provide.

As a Configuration Manager, I’m always looking for ways to improve the automation of the builds and deployments of my company’s applications. We use scripts to compile the apps, replace certain token strings with environment-specific values, and copy the new executable code out to the production servers. Ideally, we should not have to use seperate scripts when deploying to different run-time environments (development, integration test, production, etc.). We want instead to pass the target environment into these scripts, and use logic to determine environment-specific values. So I set out to create a Lookup Table to set the values according to the target environment.

I wanted to keep it simple so maintenance would be easy. I wanted it to run in a basic command shell (I use ‘bash’, but most other shells would work as well). UNIX and linux utilities like ’sed’ and ‘awk,’ and xml parsers would have done the job, but they added complexity so I stayed away from them (although I do use ‘grep’). The listing below is a simplified version of what I came up with. It takes one parameter representing the target environment, and sets 3 variables: the target server, the target database, and a process user ID. It then prints the new values to the screen for verification (an optional step). The script we actually use at work also sets target directories, service names, and website urls, but this is enough to give you the idea:

Listing 1

#!/bin/bash

# Sets environment variables based on lookup string

# Environments: DEV = Development, QA = Quality Assurance,

# UAT = User Acceptance Test, PROD = Production

ENVIRONMENT=$1

# Set server addresses, database names, and user IDs.

line=`grep ^$ENVIRONMENT <<EOF

Env Server Database User ID

— ———————— ——— ———-

DEV dev.myapp.mybusiness.com myappdev devappuser

QA qa.myapp.mybusiness.com myappqa qaappuser

UAT uat.myapp.mybusiness.com myappuat uatappuser

PROD prod.myapp.mybusiness.com myappprod prodappuser

EOF`

set — $line

export AppServer=$2

export DataBase=$3

export UserID=$4

#

# Show environment settings:

echo “AppServer = $AppServer”

echo “DataBase = $DataBase”

echo “UserID = $UserID”

Sample run:

$ ./Lookup.sh DEV

AppServer = dev.myapp.mybusiness.com

DataBase = myappdev

UserID = devappuser

$

Using the Technique

Knowing how this script works is not essential to using the technique, as long as you realize that you can expand it by adding more values to the ends of the input lines, and creating enough values with the ‘export’ statements to accomodate the new values.

>>Read explanation and rest of article >>

Helium has a great feature for writers wanting to compete for some extra money. It’s called their Marketplace. Outside publishers ask for specific articles, and offer specific amounts of money for each one. Anyone can write and submit their best article for each requested title. Once the deadline has passed, the publisher will review all the submissions and choose one or more of the best ones for publication on their own website, or even in print. Each selected article will earn the author the amount specified in the original Marketplace request. Most of the publishers utilizing Helium’s Marketplace to solicit articles operate under a pseudonym for various reasons.

As a recent example, a publisher recently requested 4 titles, all related to PC optimization. Each article will pay the selected author $75.00. You can see my submissions, as well as those of the other writers competing for these titles, by following these links:

• How to accelerate the speed of your PC
• How to prevent your PC from being infected with spyware
• How to find out what’s running on your PC (and why this is important)
• Computer performance: Why a paid-for registry cleaner is better than a free one

Recommended: to keep your PC running like it did when it was new, try Diskeeper 10.

On Helium, all articles of a given title are ranked by the other members. This ranking is a bit flawed, because complete novice writers with little command of vocabulary, spelling, and grammar carry as much weight in rating as do those with more season, skill, and talent. Nevertheless, your articles will always appear with the other articles of the same title, ordered by rank. Fortunately, the Marketplace publishers do not select based on rank, but rather based on the articles that came closest to meeting their requirements and specifications.

The deadline for these four titles is Friday, the 25th. It sometimes takes several weeks before the selected articles are announced, but when they are, I’ll report my success (or lack thereof).

Use Secure Shell (SSH) to establish safe, encrypted internet connections through a firewall. With this method, you don’t have to open additional ports through your firewall in order to access external email accounts, access usenet newsgroup servers, and multimedia streams, which leaves your internal network more secure. This means you don’t have to worry about accidentally surfing to a restricted site (which raises red flags in most corporate environments), and can access sites that have been mistakenly blocked by over-zealous monitoring software. Traffic cannot be analyzed for content by sniffers or packet inspection software because of the encryption.

Access to a Server
You will need access to a server running Secure Shell on the other side of the firewall. If you are connecting from inside your company’s firewall, you could run OpenSSH (an open source SSH server) from your home computer or that of a friend. Installing and configuring an SSH server is beyond the scope of this article, but good documentation exists for OpenSSH on its home site. Just make sure that you open a port through any router or personal firewall for SSH traffic. The default is port 22, but you can use any available TCP/IP port. Alternatively, there are some sites that will give you a free shell account on their server running SSH. SilenceIsDefeat.org will give you one for $1.00 if you use paypal (and signing up is then instant), or the cost of a $0.39 stamp if you register through the mail. Finally, if you pay for a commercial web host, many of them allow SSH connections to their servers.

Connection Settings
You also need to know a little about how you connect to the internet from within the firewall. Most companies allow web traffic through a proxy server so they can monitor the content employees are viewing, and can restrict access to sites with objectionable content. You can examine the internet connection settings for your browser. If you are set for “Direct Connection to the Internet” (Firefox), or no proxy or configuration script is set up (IE), then you probably have unfettered (but not necessarily unmonitored) access to the internet, and would only need to use SSH for privacy. You can skip down to SSH Client. If proxy access is set up in a straightforward configuration, then the proxy settings will be displayed right there. Make a note of the address and port of any HTTP or Socks proxies defined. Some companies use an Automatic Proxy Configuration Script, which makes retrieving the proxy settings a little more difficult. If you’ve already noted your proxy settings, you can skip down to SSH Client. Otherwise, here’s some help retrieving your proxy settings from an automatic script. Copy the following lines into a new blank text file and store it on your hard drive:

<HTML>
<HEAD>
<TITLE>Download a file instead of rendering it</TITLE>
<BODY>
<A xhref=”http://URL.OF.SCRIPT” mce_href=”http://URL.OF.SCRIPT” >Right-Click here and select Save As</A>
</BODY>
</HTML>

Edit this file and replace URL.OF.SCRIPT with the address of the automatic configuration script as defined in your connection settings. Save the file as dl.html. . In your browser, use File->Open to open this file. You should see a single link saying to “Right-Click here and Select Save.” Right click on the link select “save link as” or “save target as” (depending on your browser), and save the file locally. You now have a copy of the Automatic Configuration Script which you can peruse in any editor, looking for proxy information. Often a company will use a number of different proxies for different purposes - you’re mainly looking for Socks and HTTP proxies. Note the address and port of any you find. If you think you’ve found an HTTP proxy, you can test it by changing your connection settings and telling your browser to use that server and port explicitly instead of using the Automatic Configuration Script, and connecting to the internet. Just remember to restore the settings afterward.

SSH Client
The last piece you need is a Secure Shell client. This is just a program that opens and manages an encrypted connection to a server. Normally, you would use this combination to log on to, administer, and exchange data between a remote computer (the server) and your local computer (the client). We’re going to be using a more advanced feature of SSH known as “port forwarding”, which lets you direct other network traffic through such a connection. A good choice for Windows users is PuTTY, which can be downloaded freely from the Download Site. Macs and linux users will almost certainly have a good SSH client installed. I’ll be using PuTTY in the examples in the remainder of this tutorial, but the principles will be the same in any SSH client.

PuTTY Configuration
The PuTTY documentation does a good job covering configuration, so I’m just going to focus on the essentials for port forwarding. Basically, you enter the IP address or DNS name of the host to which you’ll be connecting, the port it uses, and a descriptive name in the ‘Saved Sessions’ field. If your browser uses a proxy server to access the internet, then you will configure PuTTY to use the same one(s). In PuTTY’s ‘Category’ tree (left portion of window), click the ‘+’ sign next to ‘Connection’ and click on ‘Proxy.’ If the browser had a Socks proxy configured, select that type in PuTTY. If not, but it had an HTTP proxy configured, then select that type. Enter the Proxy hostname and port that you previously noted. When this is done (or if you didn’t have to add proxy configuration), click back up on the ‘Session’ category and click the [Save] button. Then click the [Open] button. If everything is correct, you should get a new window with a login prompt from the remote system. You’re ready for the last step - actually forwarding a port or two.

There are two methods of forwarding ports through an SSH connection. “Dynamic” forwarding is easier to configure and more flexbile, but can only be used by applications that support a Socks proxy. This includes most modern web browsers, so we’ll start with setting up secure, encrypted web browsing. If you actually logged in, type ‘exit’ and press the key. Otherwise, just close the window. Bring up Putty again, select the ‘Saved Session’ you stored earlier and click the [Load] button. In the ‘Category’ tree, expand ‘Connection’, ‘SSH’, and select ‘Tunnels.’ In the ‘Source port’ field, enter 8081. This can actually be any number higher than 1024, I’m just using 8081 as an example. Select the ‘Dynamic’ radio button, then click the [Add] button. ‘D8081′ will appear in the ‘Forwarded ports’ field. Under Categories, select ‘Session’ (you may have to scroll up to see it), and click the [Save] button again. Click [Open] and log in to the remote server. In your browser, you’ll need to change your proxy settings. Make sure to write down the current settings, so you can restore them later. You are going to set the browser to use a Socks proxy (which dynamic forwarding creates for you). In IE under Lan connection settings you have to select the [Advanced] button to see the proxy configuration fields. In Firefox, they’re visible in the Network ‘Settings’ panel. Add the word localhost as the proxy host or proxy server address, and 8081 as the port. Click [OK] until you are out of the configuration screens. If you can now browse the web, then congratulations, you’re doing so in a secure, encrypted tunnel. Nobody can see what sites you visit unless they watch over your shoulder. Note: examination of your computer’s cache, log files, history, and other forensic evidence will still yield information on your surfing habits. A secure tunnel only protects the data in transit.

The second method of forwarding ports involves forwarding each port used by your network application from your local PC to the actual server running the network service you wish to access. You then configure the application to use your local machine as the server. For example, to connect to your external pop3 mail server, in PuTTY you would go back to the ‘Tunnels’ configuration screen. Add 1110 as the Source port, select the ‘Local’ radio button, enter your mail server’s address followed by ‘:110′ in the ‘Destination’ field, and click [Add]. You should see something like L1110 your.mail.server:110 appear in the ‘Forwarded Ports’ field. Once again, return to the ‘Sessions’ screen and click [Save]. Fail to do this after any changes, and you’ll lose them. Now, open your mail client. Wherever you would normally enter your pop3 server address and the port it uses, enter ‘localhost’ and 1110. In Outlook Express for example, you will find these settings in the ‘Servers’ tab and the ‘Advanced’ tab in the Account Properties screen. Once you’ve made these changes, you should be able to connect to the SSH server using your saved session in PuTTY, then retrieve and read your mail in your mail client software. Sending mail uses a different port (25),and often a different server name, so you’ll have to forward another port in a similar manner if you want to be able to send mail as well.

Pretty much any network service that uses a defined port or set of ports can be configured to work through an SSH tunnel in this manner. This includes services that your company may ordinarily block, like Instant Messaging services, Usenet Newsgroup access, streaming music sites, etc. Note that anyone with access to network sniffers or inspection software, be they crackers, hackers, or network admins, will still be able to see network traffic between your computer and the remote SSH server, they just won’t be able to tell what it is or where it goes beyond that point.

I recently wrote an article on Helium, trying for a Marketplace sale. The topic was on the benefit of static landing pages for marketing web sites. I have my doubts about getting the sale, as my article is only rated 5th out of 7 right now, but you never know. In any case, here’s an excerpt:

While dynamic pages offer a number of desirable features for web site owners like design consistency and up-to-date information, in order to optimize the effectiveness of your internet marketing website you’ll want a static landing page. Static landing pages offer several advantages over dynamic pages where attracting and retaining visitors is concerned. This is of course essential to converting web site visits to sales.

First, assuming you advertise your product or service on other web sites, when users arrive at your site after clicking one of your ads, most of them will leave within a few seconds if they don’t immediately see the information that prompted them to click on it in the first place. They do not want to fill in any forms, navigate your site looking for the information, or wait while the page is generated dynamically. Having the relevant information presented in a concise, statically loaded page will capture their attention and keep them from wandering away. For best results, this page should be customized for the advertisement or ad campaign that brought the visitor in. Thus, if you have different ads for a variety of products or services, each should link to a different landing page, optimized to deliver information relevant to that product or service, with a clear indication on what the user should do next. Many on-line marketers are realizing higher Click Through Rates this way.

Second, when content is dynamically generated, it may not always present the information your visitors are looking for. Pages are generated based on data, and if the data is unavailable (if for example, a service is not responding or a database is down), the page may not contain any information useful to your visitor. It may give error messages, or it may not render at all. Nothing will turn off first time visitors faster than a web site that appears to be broken. They’ll leave and likely never return.

Well, I still haven’t solved the earlier problems (see other posts in this category), but now I’m having a new problem. One of the Windows servers we’ve had OpenSSH running on for quite some time suddenly seems have issues. It will stop accepting connections. The message in the sshd.log is always some variation of this:

63 [main] sshd 7632 child_copy: linked dll bss write copy failed, 0×207A000..0×207CAA0, done 0, windows pid 8136, Win32 error 998

Stopping the service, with the intent of restarting, didn’t work, as the service would then not start at all. cygrunsrv -S sshd would yield the mysterious win32 error 1062, and would refuse to start, with nothing showing up in the event logs. A complete re-installation of cygwin fixed the problem, but it returned within one day. Now I find out that this server is short on memory (it’s used for some heavy-duty data processing), so I suspect that the problem is related to that. If you’re researching the same issue, check your available memory. I’ll report more details here as they develop. In the end, I’ll probably write a comprehensive article for publication on Associated Content.

Update: 12/06/2007: Some of our scripting relied on multiple successive ssh connections to a target server. The idea was to maintain as much of the scripting logic as possible on our build server, executing remote commands one at a time, each via an SSH connection. This may have caused a resource bottleneck. I re-wrote some of the scripts to do a number of things in a single connection. I also added retry logic, in case of the “resource unavailable” error. We’ll see how it goes.

I recently ran a training session teaching some Configuration Management (CM) personnel some of the basic UNIX/linux shell commands, along with some of the common ways Secure Shell (SSH) utilities can be used to move application code around during deployments. I created an outline for the class, which is reproduced below. In no way is this outline a complete reference for using shell commands and SSH for CM, but it introduces some of the basic utilities and commands that can be a part of a comprehensive CM architecture.

I) bash : a *nix shell
A) A shell is a command-line interface to an OS. There are lots of shells available in *nix (korne, bourne, etc.). bash tries to include the best features of each. Shells are related to DOS.
B) cygwin makes it work in Windows, along with most other “POSIX” compliant programs & utilities (including OpenSSH).
C) Some common shell commands (all of these work in the other shells as well):

i) cd : Change working directory.

(a) cd : By itself, cd puts you in your own home directory.
(b) cd /tmp/ftp_files : puts you in the /tmp/ftp_files directory. The leading “/” means start at the root or base of the file system, and traverse from there.
(c) cd myfiles : puts you in a subdirectory from your current location called myfiles. You could be anywhere in the file system and this form of the command will only look there for the named sub-directory.

ii) ls: List files.

(a) ls : lists files in current directory
(b) ls /usr/bin : lists files in sub-directory /usr/bin
(c) ls -l : lists files in “long” format, showing owner, permissions, sizes, etc.
(d) ls BAM* : Lists all files in current directory whose names start with “BAM”. The “*” is a wild-card.

iii) cp : Copies files from place to place, optionally with new name.

(a) cp thisfile.txt thatfile.txt : makes copy of thisfile.txt with name thatfile.txt.
(b) cp /tmp/sales.wks /home/jp : Copies file named sales.wks from directory /tmp to directory /home/jp (assuming this directory exists).
(c) cp /var/news/daily/* ~ : Copies all files (using a wild-card again) from directory /var/news/daily to the user’s home directory. The “~” by itself means current user’s home.

iv) grep : Matches a string with some source of text, often the contents of a file.

(a) grep error: *.log : Searches all files in the current directory whose names end in .log for any lines containing the text “error:”. If it finds any, it lists the file name along with the actual line of matching text.
(b) grep -i virus ~brian/* : Searches all the files in user Brian’s home directory for any file which includes the term “virus”. The “-i” switch makes the search case-insensitive. The “~” followed immediately by a user name is short-hand meaning the named user’s home directory.

[click to continue...]

The “Could not create an instance of the CmdLib object. Please register the Microsoft.CmdLib component.” error message was because of certain web server extensions that weren’t installed. I think the SQL Server Reporting Server needed to be installed. In any case, we’re past that now.

Current problem: we’re running the sshd service with a Domain Admin ID. This works, but poses a security risk. I’m trying to get a test installation working where the service ID is a Domain User but not a Domain Admin. The service starts, but anyone connecting to it is dis-connected immediately after authentication. The debug message (running the server with full debug messages logged) states “fatal: setreuid 14153: No such process.” So again, looking for any pointers. I’ve been all over Google about this, nothing that seems to apply yet.

Recently it was decided at the large State government facility where I work that Secure Shell (SSH) would be used to facilitate the deployment of application software to the servers. This approach has a number of advantages. All data transfers are encrypted. Key pairs can be used to automate the authentication, so the entire transfer can be scripted with Shell scripts. Plus, with SSH’s ability to execute remote commands, command-line utilities on the target server could be utilized to stop and start services and web sites as necessary during the deployments.

Since most of the target servers that host our applications are Windows servers, that meant installing a 3rd party SSH server. The State opted for OpenSSH, the Open Source SSH implementation. To get that to work in a Windows environment requires Cygwin, a “linux-like” shell environment that runs under Windows. Getting this to work in a default environment is a snap. Getting it to work in a complex environment which includes Active Directory, domain controllers, and Group Policy Objects has proved to be quite a challenge. We are making progress, and I’ll probably write up the entire process in a “How-To” article in the near future. In the mean-time, I’m struggling with one weird error trying to stop and start web-sites.

We are using a command-line utility named iisweb.vbs to stop and start the services. The ID we are running the script under appears to have the necessary privileges to use this utility (being a Domain Admin), but when it executes, it errors out with the message “Could not create an instance of the CmdLib object. Please register the Microsoft.CmdLib component.” The funny thing is, this was working until yesterday, when the server teams re-built the server in order to re-partition the drives. Any help would definitely be appreciated.

Jp

Next post in series…

While waiting for Associated Content to publish the Charity Music article (something they’ve now agreed to do), I wrote a quick little overview of SSH and published it on Helium, where it’s now rated #1 of 2. What is SSH? The nice thing about Helium is that your submissions are published more or less immediately.