Why would I need to convert?

• You are tired of using entities for foreign characters. Hate using things like this ŵ (← very common character… uhm… )
• You are going to merge your web site with another website. (Which has a different encoding.)
• You want to write in more languages. (I’m talking about languages as in the languages people speak NOT programming languages.)
• When generating emails and sending data you have problems with weird signs.
• You are mad because ISO-8859-1 doesn’t contain an euro sign (€). (Isn’t that quite funny? I mean it’s encoding especially suited for the west of europe. Yea, yea € I know. But still.)

There are more reasons. Feel free to add those in the comments.

The biggest reasons “back in the days” to use ISO-8859-1 was that it had slightly better support in older user agents. Well those old agents are pretty much gone by now. Who still uses Netscape 4? Hands up! Anyone? Well, that’s what I thought.
All in all UTF-8 is, in the long run, more fit for the future.

Don’t go to hell

So what do I mean by “Encoding Hell”. Well those who has been there know what I mean. It’s when you just switch encoding without properly converting your data and you end up with letters looking sÃ¥ här (that’s the word “så här” meaning “like this” in swedish.) I got this nasty letters by opening a file with UTF-8 encoding in my text editor using ISO-8859-1. And guess what? Opening with the right encoding doesn’t switch the letters back. They are stuck looking nasty. I’m in hell!

Well this was only one file and sure you can change the letters manually or by using some fancy search/replace-command. But think about if you made this mistake with hundreds or thousands of files? Not nice. Big hell!

Do it right from the beginning

The best thing is to avoid Encoding Hell all together. Choose the right encoding from start and you are safe.
But if you must change. First thing. Be careful! If you are careful you will save time. Make up a converting-encoding-plan first so you know exactly what you are doing. Just don’t start testing. My advice is to always start with a backup. Backup everything you want to convert. If you have a database connected to your webpages don’t forget to back up that one as well.

Converting your files

Don’t even think of converting your files by using your text editor. It usually turns out nasty. There are tools to help us out with this. You can use the terminal/the command prompt to convert things. Me, myself, I have tried out “Encoding Master” which I think is simply brilliant. Easy to understand and it converted everything very nicely. It’s also supported by both PC and Mac.
Don’t forget to add/change the encoding in the headers of the webpages too.
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">

Converting your database

Be careful here. Don’t start adding UTF-8 encoded data into your old database. Make sure it’s converted and pretty first. Otherwise you might end up with a messy mix and bang! You are back in hell!
To change the encoding of your database you must change the encoding of everything. The database itself. Every table in the database and every column that includes some sort of text-value. I’m talking about data of type: char, varchar, tinytext, text, mediumtext and longtext.
Since I lately been working with PHP and MySQL the following will apply to them. If you are working in other languages and other databases you probably need to do some further research to see what is the right methods to use.

To change the databse itself use the following query:
ALTER DATABASE my_database charset=utf8;
To change a table use :
ALTER TABLE my_table charset=utf8;
To change a column in a table use:
ALTER TABLE my_table MODIFY my_column TEXT CHARACTER SET utf8;

Make sure everything in there uses the character set utf8 and collation utf8_unicode_ci. Ok?

Encoding in PHP

If you are coding in PHP you might have problems with the MySQL and PHP talking UTF-8 language. Solution to this which worked for me was to send the following query:
SET NAMES 'utf8' COLLATE 'utf8_unicode_ci'; This must be done right after connecting to the database.

From Simon comes this handy tip to those of you who are using MySQLI together with UTF-8:
(With error-handling included as a bonus…)

$mysqli = new mysqli(DB_SERVER, DB_USERNAME, DB_PASSWORD,
DB_DATABASE);
if(mysqli_connect_error()){
throw new Exception('Connection failed
due to the following error: '.$error);
}
if (!$mysqli->set_charset("utf8")){
printf("Error loading character set utf8: %s\n",
$mysqli->error);
}

It could also be a good idea to add the code below to the beginning of the PHP-files.
mb_language('uni');
mb_internal_encoding('UTF-8');

Last but not Least…

Always research carefully before changing your encoding. Especially if you are a converting-virgin and are changing the encoding for the first time. Never rely solely on one tutorial. Not even this one which, I need to admit, is not covering everything. This is pretty general info. Hey, you can write a book about encoding and the struggles with converting. So remember to look around. Ask people. Start with a backup. Always. And, again, be careful!

Happy Encoding!

/Ida

Why is validation not enough?

If your website passes the validation check it means that the code you’ve written is valid. It does NOT mean that:

• - your website is accessible
• - your website renders well on all devices and browsers
• - your website is optimized
• - it doesn’t even mean that your code is well written (No it doesn’t!)

So let’s sort this out by asking what HTML actually is.
According to Wikipedia HTML “provides a means to describe the structure of text-based information in a document” I think that describes it pretty well.

Let’s take a look at the following code:

<div id="h-123">
<span class="span-b">
<span class="big-first-letter">M</span>y header
</span>
</div>

The code above is perfectly valid according to W3C’s Validation Service. But let’s be honest. Do you think this is good descriptive markup? Do you think this markup would be considered semantic? Is it optimized?

Instead consider this code:

<h1>My header</h1>

Much shorter and much much better.

So I think you get my point. But if validation isn’t enough how do you write good HTML? I don’t know any good application today that can actually give you a good answer to if your HTML is good or not. If you do please feel free to post a comment below.

A few tips to help you write that “good” HTML

So I’m not gonna give you a perfect easy-to-follow answer. But I’m going to give a few tips which help me when I write HTML.

• – Think about what you are describing and not what it should look like. A header should not be a h1 just beacuse you want it to look big. In short: HTML is not about appearance!
• – Write (ALL) your structure (i.e. HTML) before trying to achieve whatever design you have in mind. If you do that you will not add the wrong stuff to your code like unecessary classes (known as classitis) and it’ll also give you a good protection against divitis.
• - If you tend to forget about the existence of certain elements have the HTML 4.01 / XHTML 1.0 Reference open while writing.
• - Use attributes the right way. Like alt and title which people tend to mix up. (Alt is the caption and title is the description.) A list of attributes can be found here.
• - If you know you have a problem and tend to use to many tags. Try to think less is more. Keep it clean! (And take another look at the header-example above.)
• - After adding some style to your website go through the guidelines. Yea, I’ll admit it. I don’t do this myself that much. But every once in a while try to force yourself to at least skim through the guidelines. And if you like to say that you build truly accessible websites I consider knowing these rules to be a must.
• - And as a last step. Validate.

Finally remember that if your HTML truly is “good” there will be no problem for nobody when it comes to interpret your code. That goes for both screenreaders and search engine spiders.

/Ida