Apache has a very decent authentication scheme. In Apache you define what are called "Authorization Realms". You can define these in your apache configuration files on a directory or site, or you can define them in a .htaccess file. Each of the Authorization Realm specifications will point to a password file and will look like this:

This file defines a few things, it defines which users to allow, where the passwords are stored, and the name of the Authorization Realm. The name of the authorization realm is important, because you can have multiple layers of authentication. For example, you could have a folder that any valid user can access, and a subfolder which only 2 specified valid users have access to. If you ensure that the AuthName is set the same, the user will only have to enter their username and password once, whereas if you have different AuthNames, the user will be required to pass into each realm separately, even if the password file is the same.

Important Security Note

Any time you use Authorization through Apache, you should ensure that your users are connecting with HTTPS, otherwise the passwords will be sent in plain text, and anyone listening in the middle could see and capture them.

I need to make some decisions soon about MortalPowers.com. This includes questions about the appearance, functionality, the focus of my time, and what sort of browsers and codecs I want to support. To answer these questions, I'm going to try to go back to the original question, which is asking myself what are my goals, and why am I building this site.

What is my goal with this website? My goal is to increase traffic, to provide a place for my development projects to live; grow; and be shared, and finally to meet the needs of my visitors.

The last goal is going to be one of the most central, because it will inform the first two. But to answer this question, I need to know as accurately as possible, who are my visitors? I'm going to use the last month as the window and breakdown my traffic from multiple source (Google Analytics and AWStats primarily).

Who visits my site?

Browsers

Right out of the gate, 8% of my visitors will be unable to view any HTML5 videos. At this point, I'm comfortable instantly dropping the 8% of my visitors that still use Internet Explorer. This is partially a philosophical decision, but it's also in part because most of the people that come using that browser will have no interest in anything I have to say.

Only about 3% of my Firefox users will be unable to view my Ogg Theora videos. This indicates that Firefox users tend to update their browser relatively frequently. This is great, because Firefox and Mozilla have committed to support to a lot of the technologies I want to use and support.

Operating Systems

It seems that a major plurality of my users (45.5%) still use windows to access my site. This is acceptable to me because Windows users can still be interested in Open Source, in software development, or even interested in trying or understanding Ubuntu. I will try to do what I can to support these individuals. As an interesting tidbit, Windows use ends up breaking down as follows for me: 43.9% use Windows 7, 39.22% use Windows XP, and 14.03% use Vista, the rest use Windows Server 2003.

Screen Resolution

Less than 7% of my users have screens of 1024 width, everyone else has 1280 or much greater.

What this means for Video

One of the goals that has most recently bubbled to the surface for me is the idea of supporting my video gallery on mobile. I'm continually disappointed with YouTube's mobile site because their video collection is a subset of their full video gallery. Looking at the video support offered by the users of my site, I can cover almost all of my desktop users with Ogg Theora, and until WebM becomes a universal standard, this will be the universal standard. I'm not anticipating WebM to become available across my user base until at least 2012, but time will tell. On mobile, the only option is to use .h264. Based on these competing values, I'm going to attempt to have 2 encodings for each of my videos. I'm not sure yet whether this will mean I have a mobile version of the gallery, or whether I will try to use a single HTML5 video gallery with multiple sources, and I'm definitely open to feedback.

Over the next few months and years, I'm anticipating that mobile usage of my website will grow from the lowly 1% to 20%-50%. Google itself has declared a "mobile first" strategy, and the human workflow of using a phone seems to be much more natural than the bulky process of using a laptop or desktop computer, despite the speed, cost, and additional capabilities provided by desktop computers.

I hate the idea of supporting the .h264 standard, but until WebM is available, I'm going to have to make a decision in favor of my users, rather than in favor of my philosophy that Android and iPhone and Internet Explorer seem to have already rejected. It still makes no sense to me why the mobile phones and Internet Explorer don't at least offer Theora video decoding, since the software is free (as in speech) for them to implement.

Someone asked for an example of Apache Virtual Hosts configuration. All of my virtual hosts have the same format as the following two examples:

Types of Virtual Host Configurations

Name Based Virtual Hosts

There are two main types of virtual host configuration. The first called "Name Based" virtual hosting matches the examples above. In this case, the domain name provided as part of the request determines the site that apache matches. Name based hosting does not work with SSL because Apache doesn't know the domain name requested until AFTER the encryption negotiation is complete.

Address Based Virtual Hosts

The second type of virtual host is address based hosting. This will only be used when the server has many IPs. The server uses the interface (server ip or port) of the incoming request to determine the site to respond with. This is almost always used to establish SSL hosting in combination with name based hosting.

Example address based virtual hosts:

Dynamic Virtual Hosts

The third type (which is new to me) is mass or dynamically configured hosts. These hosts use the IP or Domain name to determine the DocumentRoot of the site. Read the Apache spec here. Example of what would go in your httpd.conf below:

The %0 when used with VirtualDocumentRoot replaces %0 with the domain name.

For the longest time I have been perplexed by the issue of an html div tag and its children being affected by a float present in the grandparent element. The behavior I was seeing was that whenever I would float any element inside of my div, it would be pushed down to below the floated elements in the grandparent element. This was definitely not desired, as it made the vertical location of the new floated element unpredictable. Researching this issue lead me to very few details on the topic.

In the end, going back to the W3c specifications gave me my answer. Specifically the section on Block Formatting was useful. It seems that what I really wanted was to set up a new block formatting context. To do so, I had to set the overflow property such as overflow:hidden;. Doing so set up a new context, and allowed my elements to float as desired., while ignoring the grandparent context.

As a strange and useful side effect, I was also able to remove the margin-right:200px; from my main column CSS, as apparently that part is handled just fine by the overflow:hidden;. Testing in Firefox and Chromium show that this works great and looks just fine. If someone wants to test it in another browser and let me know how bad I have broken my site, I'd appreciate it, :).

One of the most common reasons to build a tool for an open source developer is when the developer him or herself wants the tool. This is the case with Reddit Imager, a tool designed to improve Reddit. The concept is simple, whenever there is a URL or image format we recognize (very few right now), replace the tiny thumbnail with the blown up version, maximized to the screen width. I launched this extension on the Chrome Extension Gallery because I wanted to try out the gallery at least once, despite my dislike of giving Google control. Now the extension has a couple users, and I want to make sure they get the features and capabilities they want.

What's new in 1.1?

Reddit Imager 1.1 released today adds an options page where users can specify their mode. Normal mode automatically replaces matching thumbnails with their blown up versions. I also added activated mode, which waits for a user to try to click on the thumbnail before blowing it up. I find this feature a little confusing because there are very few thumbnails we are actually capable of blowing up, but it was requested and I wanted to try it out.

Let me know if you think I should switch default modes, add other options, or add some type of visual indicator for which images can be blown up on the home page.

Report issues or feature requests here: http://github.com/PeEllAvaj/redditimager/issues

One of the tools that I end up using a lot of the time is GZIP detection. It's really useful to find out if a given URL uses GZIP. It's easy to do this with Firebug, or with Chromium's Developer tools, but these tools aren't always available. GZIP is important to save bandwidth for content sent over the internet, and detection is the first step to understanding.

Check out the GZIP Detector.

Something I have been struggling with for the past few months is the concept of building additional backlinks. Backlinks are links from other websites that refer to your content. These links will bring visitors as well as authority to your site.

The best guide I have found on this comes from Matt Cutts at Google. He breaks the techniques down as the following:

• Build Great Content - Filling your site with quality content that is useful or interesting will bring users and traffic.
• Controversy - Discussing or making assertions causing or surrounding content can attract links, but can end up hurting your relationships with other people or sites.
• Participate in the Community
  • Answer Questions
  • Help Other People
• Original Research - If you spend time to produce research on or to understand a new topic, or an old topic with a new approach, your content will be unique and will attract new users and traffic.
• Newsletters - Make it easier for your existing content to be delivered to users.
• Social Media - Connect to the people in the places where they spend their time.
• Present or Give Speeches - Giving a speech or lecture as an expert on a topic will drive traffic to the web content you have produced.
• Create How-Tos - Very specific how-to's can help save you and others future time, and can match up well with people looking for these answers. Having a resource that no one else has is extrmely useful.
• Make a Product or Service - Solving a problem will help bring people to you, as well as contribute back to the community and the world.
• Have Good Site Architecture - Being able to crawl and bookmark your site ensures people will find it easy to link to your site and your content.
• Make Videos - Simply talking to a camera can add personality and value and entertainment to your visitors.

Whenever you have form data, it is important that you communicate with users what is happening with this form data. The first degree of functionality that can be provided to users is notifying them when they have modified a form and they attempt to leave the page without saving the data. The further degree that the user interaction can be taken to is auto-save, but here we will limit this article to user communication.

There are a lot of ways to achieve this type of user communication, and I'll just be covering one of these. The first step is to build a change detection mechanism.

From there, you should hook up the page events and change detection to the forms that should be saved before leaving the page. This method currently assumes that users must submit the form and leave the page in order to save the changes.

The final piece that you need to do is hook up your form submission to allow the unload of the page. You do this because the form submission is the safe way of leaving.

With those changes, your users will now be alerted before attempting to leave the page with unsaved data, hopefully saving your content from certain destruction.
comments | permalink

Today I was browsing the Google Webmaster tools, and it notified me that I had a broken link to "/img/.html", which was odd. I checked out the pages that Google reported referring to it, and was confused to see that none of the links on the page lead to this url. I did a scan of my site for this string, and found it hidden in a piece of javascript.

The background was that I have a very simple piece of javascript that allows keyboard navigation of my photos. I was lazy and built it with the following PHP:

As you can see, when this variable is empty (It is still sent to the browser by the PHP), the javascript check fails and disallows the event. What was odd though was that because PHP was still outputting window.location = "/img/.html";, Google tried to crawl the page, and found that it didn't exist, reporting a broken link.

This level of sophistication is astounding, even though it didn't work. I'm going to need to be more careful in the future with URLs that I put in javascript. Perhaps this has some sort of impact on keywords used in Javascript as well?

The new Froyo API has a few jokes and curious pieces. I am most interested by the following methods from the api:

• ActivityManager.isUserAMonkey()
• Log.wtf()
• GRAVITY_DEATH_STAR_I

If you aren't doing any sort of analytics for your website, you should be. Analytics is useful because it tells you what people are consuming on your site, and how they are consuming it. In theory, most web pages are designed to be consumed by visitors, and this means that you need to understand your visitors in order to build a quality site that meets their needs. Understanding your analtyics and your users should also have the goal of growing your site traffic by responding to the things you learn.

There are two great tools for tracking usage of your site. The first one is called AWStats and is an open source project. The second is called Google Analytics, and is a proprietary service provided by Google. I recommend using the combination of these two tools to get a more complete picture of your site.

You need to use both tools because each has their strengths and weaknesses. Google Analytics provides spam and robot-free data, it comes through the lens of Google. This means that users that don't have javascript won't be included, users that block Google Analytics won't be reporting, and there are many cases where Google under or over reports the amount of traffic you are seeing. Unfortunately there is no way to know how or why they are changing your results. Awstats is great because it provides exact data on the responses your server is sending, unfortunately this data is colored by robots, spammers, and doesn't always draw a complete picture of the experiences of users on your site.

Use BetterAwstats as a drop-in front-end replacement for Awstats. This tool is great for managing multiple sites (data can be combined, or browsed more easily through the menu on the left). It is also great because it allows you to more easily see long-term trends such as multi-year month-by-month changes.

Server security is vital to anyone running a web server of any kind. For brevity in this article, I am going to call people that are attempting to break or break into your server "hackers". There are lots of attack vectors hackers can use. Some hackers are attempting to attack a specific server, whereas some hackers are attempting to exploit general vulnerabilities and do what is called "door knocking" where they check large numbers of servers for known vulnerabilities.

One good way to start checking your own server is with a tool called nitko. Nitko is a tool that is easily installed on ubuntu from the repositories, and is run with nitko -h <server>. Running nikto against your server checks for several thousand known vulnerabilities and misconfigurations on your server. It will also tell you if the versions of any of your critical applications (such as php, ssl, apache) are out of date. For example, running nitko against this website reveals the following:

- Nikto v2.03/2.04
---------------------------------------------------------------------------
+ Target IP:          67.207.147.101
+ Target Hostname:    mortalpowers.com
+ Target Port:        80
---------------------------------------------------------------------------
+ Server: Apache/2.2.12 (Ubuntu)
- /robots.txt - contains 3 'disallow' entries which should be manually viewed. (GET)
+ No CGI Directories found (use '-C all' to force check all possible dirs)
+ OSVDB-0: ETag header found on server, inode: 590191, size: 132, mtime: 0x484f2a92b9380
+ OSVDB-0: GET /index.php/"><script><script>alert(document.cookie)</script>< : eZ publish v3 and prior allow Cross Site Scripting (XSS). http://www.cert.org/advisories/CA-2000-02.html.
+ OSVDB-3092: GET /sitemap.xml : This gives a nice listing of the site content.
+ OSVDB-3092: GET /test.html : This might be interesting...
+ OSVDB-3268: GET /icons/ : Directory indexing is enabled: /icons
+ OSVDB-3233: GET /icons/README : Apache default file found.
+ 3577 items checked: 7 item(s) reported on remote host
+ End Time:        (195 seconds)
---------------------------------------------------------------------------
+ 1 host(s) tested

These results show me 7 items I should look at carefully on my server. The only thing I might want to change is that I have allowed the default Apache installation to share the /icons/ folder and make it an browsable directory. This might be a concern for other sites, but it doesn't really have any security impact on my system.

While developing my CSS Destyler, I came across an error in my console stating "Illegal Access", the relevant line looked as follows:

After some initial exploration, I determined that the issue was not related to the access of the localStorage, as "illegal access" would make me think. Instead it was related to the JSON parse statement.

For some reason, JSON errors with "illegal access" whenever you try to parse a null. The solution was to ensure that the item passed into JSON.parse was not null.

When writing earlier versions of HTML, it was appropriate to specify any extra namespaces as attributes in the html tag. In HTML5, there are no longer traditional namespaces. Instead, standard namespaces such as SVG are automatically recognized in the children tags of an SVG element. This means that if you had a facebook connect integration on your site, with a namespace declaration:

You should replace this line with:

This change will help your HTML5 to validate, and shouldn't affect the functionality of your webpage, unless you are doing something unusual such as having integration with non-browser XML parsers that need the namespace declarations.

As modern web development methodologies continue to improve and optimize our workflows, one of the workflows that has risen to the top is a development deployment model using git.

Making it Secure

Before you do any of these steps, make sure you add the following to your .htaccess, and that URL Rewriting is working in your apache setup.

Without doing this, anyone will have read-access to all of your code, and your entire source history.

Use git

This guide assumes a lot of knowledge about source control, management, and is not a guide to git. This guide assumes you can figure those parts out, but that you want to know more about one specific workflow that works very well in my opinion.

The main idea behind this workflow is that you will have a central repository, a development version of the software, and a published version of the software.

Convert something else to git

If you want to keep any history, the first step is going to be to convert the source control history to git. If you are using cvs, you will need to convert to SVN, and then to git. If you use SVN, simply use git to do a git svn clone of the repository somewhere on your local system. From there I recommend you move the .git folder to the remote production server, and make a copy in the development folder. If you go into each of these folders and type git status, you will see any differences that have not yet been committed to the repository, but you will be ready to make and receive changes.

Now that you are setup, time to develop

From there, things are relatively simple. Simply make any changes you want in your development environment. When you are finished with your changes, and they have been tested to your satisfaction, commit the changes, and push them to the repository. You can then enter your production server, and pull the changes from the repository. You can also simply pull directly from the development server, but organizations I have worked with tend to like a 3rd copy of the source, somewhere separated from the hosting environment.

If you want to, you can make emergency or urgent changes directly on the production, following any existing workflow you have for ensuring something is not broken. When these changes are complete to your satisfaction, simply commit and push them, and do a pull from the development environment.
comments | permalink

Google Analytics continues to prove itself as one of the top and most important tools for website analytics today. Recently Google added a new feature for those that use both Google Adsense, and Google Analytics. This feature is the ability to cross-reference data between these two applications. What this means is that not only can you see what visitors are coming to your site, and what they are viewing, but you can cross-reference this data with how much money they are providing you.

As an example, if we look at mortalpowers.com we will see that Google Chrome accounts for around 30% of the traffic. What is odd is that over the past 3 weeks, 100% of adsense revenue has come from Chrome. This strange pattern is again repeated for Linux. Linux makes up around 40-50% of traffic, but ends up providing nearly 100% of adsense revenue.

Keep in mind that for this site, sample sizes are very small. From this type of data and with a larger sample size, it is clear that these new features are going to help those seeking to monetize their web presence and content greatly.

Internet Explorer usage continues to decline, losing somewhere between a tenth of a percent and half a percent each month to better browsers such as Firefox and Chrome. My prediction is that this is going to slow and eventually come to a stop. The problem is that there is a solid percent of internet users that have no idea what a brower is, and don't want to know, and will probably never learn. This block of users will never be persuaded by functionality or speed or new features.

In response to this inevitability, I propose people that write about or involve themselves in the Linux, or even in the general power user populous, stop supporting all versions of Internet Explorer immediately. I will admit that my website still has around 20-30% usage of Internet Explorer, but I am coming more and more to the opinion that those who continue to use it, probably wouldn't understand anything I'm writing about. This leads me to believe that this traffic ends up being people that are confused, as well as web spyders disguising themselves as a normal user, artificially inflating the numbers for IE.

Benefits of abandoning IE

By abandoning IE, it makes web development and testing easier. Development can focus on the standard DOM, and on leveraging some of the more advanced functionality of the web, such as new request types, as well as little things like rounded corners, or every-other CSS selectors.

Abandoning IE as a web developer or publisher is going to send a message to Microsoft and indirectly to uninformed users about their computing choices. When a percentage of the websites a person visits in a day stop working, they are going to ask why. If the answer is that their browser is broken (which it truly is), they will be much more likely to make a switch. The message it sends to Microsoft is that shipping a broken browser that is out of touch with the modern internet is going to hurt their users. This would also tell them that if they want to ship a working browser (which is what Apple and the open source community had to do in the past), they are going to have to adopt the features and usability of the other browsers, which in this case would be primarily based on standards and extensibility.

Problems with abandoning IE

By abandoning IE, you may make your website unusable for users that don't know, or can't upgrade (some corporate offices still require everyone to run IE6). This could also decrease your traffic. There is also a chance that Microsoft will just ignore any sites that do this, which is a definite probability based on their past behavior.

Summary and Philosophy

In theory by abandoning IE you will be able to focus on innovating for your best users, rather than hacking and making workarounds for your worst.

The W3C recommended way of targeting mobile devices is for stylesheets to use the media attribute. Unfortunately, this attribute isn't used by either the Android browsers or the iPhone browser. Instead these browser report that their media is of type "screen", this is the standard used by desktop browsers as well.

There is a trick that is possible still using media detection that selectively applies the styles based on the screen width. The following code is an example of two stylesheets. The first should be applied on desktop computers, and the second should be applied on Android and iPhone.

Find additional information and tips at http://www.rkblog.rk.edu.pl/w/p/optimizing-websites-iphone-and-android/

One of the recent advancements in the HTML and page processing done by search engines, is the addition of the rel="canonical" link tag. This link tag is added to the head section of the HTML, and it indicates to search engines what the preferred URL of this page is. This is useful because with the advetn of dynamic pages over the past 10 years, we are currently in a place where often the same contents are accessible in many different ways, with many different options. This means that without this additional piece of metadata, a search engine can't tell if url?s=newest is the same page as url?s=oldest. This would be an example where the URLs represent the same content. It also can't tell if url?p=dyanmic is the same as url?p=history are the same page. This example could be a single PHP URL that shows different pages based on the get variables.

The solution is to add the following line of code to any URL that is addressed in multiple ways, or has multiple URLs for the single set of content.

Google is the first search engine to support this, but because it makes things easier for the search engine's processing, it will likely soon be supported by all major search engines. This technique can even be used cross-domain if you have content hosted and served across multiple domains.

The game I have been working on for a few months has now been packaged and lauched in its first incarnation! The instructions for download, files, etc, are all located on it's new site at hexslayer.com.

New Website

I decided to launch the game at its own URL, as a lot of the things that I put onto mortalpowers.com wouldn't relate to people interested in the game. I have developed my own blog software in the past, and used Wordpress and other options, and read a lot about some of the newer options. For this project, I decided to use Drupal, as my research has revealed it to be an extremely powerful (and albeit complex) system. I'm trying to stick with only the core modules, and major modules for the system (xmlsitemap, images, uploads) for maintainability purposes. If I adopt some unknown module it will be harder to stay up to date with security/other patches, and I might risk losing those modules if/when I upgrade to Drupal 7.

So far, drupal has been a very interesting and decent piece of software. For the type of CMS I was looking for, it is meeting all of my needs. My goal was to be able to create pages, posts, comments, and upload files and images, and to have a decent looking theme. Feel free to take a look at the HexSLayer site and let me know if I have met these goals.

The site is also on Github. One of my goals is to continue doing things in as open a manner as possible so that if someone else wants to contribute back to me, or learn from what I have done, they can.

Debian Packages

I have spent most of the evening re-learning Debian packaging (I use Ubuntu), and figuring out how to include the packaging component with the normal source code in a way that won't make it more difficult to compile/package for other environments. The solution I found was to create a packaging folder with scripts to generate packages/distributions for each of the platforms/environments I support. This seems to be working well, but I only have the Debian .deb file so far. The package includes nice features like menu and .desktop entries, which makes the game easier to launch and remember, rather than having a raw folder of .pyc files somewhere in your home directory.