What is TRIM and why do SSDs need it?

Note: This was originally written as an email explaination. I didn't bother to clean it up much before posting it.

The reason for this whole SSD TRIM issue is due to differences between SSDs and HDDs, and attempts to remain insanely backwards-compatible.

Since I'm writing this for everyone, please don't get upset if I'm telling you something you already know... to the late 1980s!

PC IDE hard drives originally used a type of addressing system called "CHS". This stood for "Cylinders, heads, sectors". This described the physical location of data on the disk.

In detail:

• Cylinders were the individual tracks on each disk. Unlike a CD or phonograph where there is a single spiral track going from the center to the edge, hard drives have closed circular loops, similar to the layers of a onion when viewed from a flat cross-section. Since each platter of a hard drive has the same general layout, e.g. Track #1 is directly above and/or below Track #1 on all the other platters, it was best to describe "Track #so-and-so on any side" as a cylinder.
• Heads. Each hard drive platter has it's own set of read-write heads, generally two per platter (one per side), although a few odd drives only used one side of each platter. This value instructed the drive which platter and side to look for data on. 
• Sectors. The smallest unit of data on a hard drive is a sector. These can be any number of sizes, from 128 bytes per sector, up to 16K per sector. The most common value for this is 512 bytes per sector, as that's what PC floppy disks used. 

However, this is a complicated way of accessing data on a drive, so this was replaced later by LBA - Logical Block Addressing. LBA reduced the former three dimensional addressing system for a one-dimensional block address - basically all of the drive's sectors are now just one very long tape, from the perspective of the BIOS and OS, and the physical layout of the sectors on disk no longer matters. The big thing, though, is that hard drives still used 512 sectors, or at least claimed to, using some internal trickery that's completely invisible to the outside world

Modern SSDs (and many modern HDDs) will report their physical sector size as 512 bytes, for the sake of backwards compatibility (there are some BIOSes as late as 2008 that will fail to recognize any hard drive or SSD that reports it's sector size as anything other than 512 bytes.) However, their actual physical sector size is generally 4096 bytes.

Standard EEPROMs erase their data one machine word at a time (8-32 bits, depending on the data bus width.) Flash EEPROM, such as that used in SSDs, is erased in "pages", which are generally 4096 bytes. This means that to change a single 512 byte sector stored on a SSD with flash memory, the SSD's controller reads the entire 4096 byte page, swaps out a single 512 byte chunk with the new sector data from the OS, erases the entire page (as you can't erase smaller than this on flash memory), and then writes back 512 changed bytes, plus 3584 bytes that was unchanged!

And that leads us to another issue: Flash memory cells can only sustain so many erasure cycles. They eventually reach a state where the stored charge is too much for the high voltage erase pulse to erase, and the old data is permanently burned in.

So, SSDs don't just do this read/modify/erase/write, you see, they try their damnest to NOT erase. So instead, it writes the logical sectors to ANOTHER PAGE on flash, mark the "old" location as "unused" (it won't erase it right away, but it will schedule it for erasure at some later time, like when the OS hasn't made a request in five minutes, or something of the sort.) This process is known as "wear levelling" and is intended to extend the useful life of SSDs.

Also, filesystems like FAT32 and NTFS don't store file location on disk by sector address, they group sectors in "clusters", which are always a power-of-two count of sectors. It's a trick to help reduce filesystem fragmentation.

The problem further then is that, e.g. you write a bunch of small files to your SSD, they might use one or two logical sectors of an eight sector page. But your SSD is smart, it knows that later if it needs to combine these two "used" sectors with two "used" sectors from a "distant" part of the drive, or some other combinations of the sort, into a single flash page, if it helps the SSD avoid erasing any one flash memory page too many times.

And now where TRIM becomes /sorely/ needed: Because file deletion doesn't do anything to the actual disk locations where the file data was actually stored, and simply removes the file's information from the filesystem's "table of contents", the SSD doesn't know that the data was actually deleted. It doesn't know that it can stop copying that data around each time it has to remap the data on the flash memory; sometimes it'll try to combine two pages whose combined "used" sector count is higher than 8; this means that the leftover sectors are then merged with another block, if there's still leftovers, it keeps merging until there aren't any more leftover "sectors". But this means that the flash memory is being written more than once in order to complete a write of what the OS thought was just one single sector. This problem is known as write amplification and it is the cause of the poor performance of older SSDs, and also quickly reduces their useful lives.

But TRIM solves this. TRIM provides an industry standard ATA protocol command for instructing a SSD that a logical data sector has indeed been deleted by the user or OS, and that the data no longer needs be meticiuously copied and maintained (and can safely be erased and treated as empty.)

I'm still alive!

Wow, I haven't made the first post in 18 months.

But, hey, that's okay, let me just remind people I'm still alive, and make what is likely my one and only post for 2013.

So I'll just leave you with this awesome two-part collaboration:

Part 1 (from Minute Physics):

Part 2 (from Vsauce):

Copyright (or: Why It's Broken)

First post in over a year, guys, I'm still alive.

Today's subject is on copyright. Namely, on why the current copyright system is broken. This post will focus on the copyright system in the United States. The situation may or may not be the same in your jurisdiction, so readers outside of the US, please keep this in mind.

First, a little history: Starting with the first passage of the Copyright Act of 1790, the duration of copyright was set at fourteen years from date of a work's creation, and if the author was still alive at the end of that term, he or she could apply for one more extension of fourteen years. The act was simple, it only covered books, maps, and charts, only afforded protection to works created by US citizens, and was geared for utilitarian purposes, namely to foster innovation by making blatant plagiarisms illegal. It also specified a statute of limitations on infringement to just one year.


As time went on, new copyright Acts were passed extending the duration of copyright with each new Act. New types of media were included as copyrightable, and new requirements for attaining copyright protection were added (such as requiring the encircled "C" icon, or the words "All rights reserved.")


Today, works created by individuals are protected for the life of the creator, plus seventy additional years. Works created "for hire", that is, under the banner of a corporation, like the vast majority of movies, musical releases, and video games, are protected for ninety-five years from their first publication, or one hundred and twenty years from their creation, if unpublished.


What this means is that everything created throughout your lifetime will almost surely never enter the public domain during your lifetime. This means that rightsholders will continue to charge for access to content long after any significant entertainment value is gone, or more likely, use copyright to bar access to works that the public has a demand for while the rightsholders refuse to create any further supply.


Let's consider this example: The original Super Mario Brothers game, as released in 1985, were the original Copyright Act still in effect, would enter the public domain in a few months' time. Unfortunately, by the time this game is released into the public domain, the majority of individuals interested in the game for entertainment purposes will either be dead, or much too old to actually enjoy it anyways, leaving the only remaining use when that time comes to be that of academic treatment.

Now, let me remind you that the vast majority of innovation is iterative, not revolutionary. Consider the Call of Duty franchise, whose last truly innovative release was Call of Duty: Modern Warfare. The innovation here is ingrained in the title itself - "modern warfare", the switch from a World War II setting to a contemporary one. All subsequent releases did not bring anything new nor revolutionary to the table, instead opting to iterate upon the previous works.

Copyright prevents others from attempting to try out minor ideas or iterations of an existing work, that is, adapting an existing work to create a new one. Creative mashups such as "Mari0" are in all likelihood in direct violation of copyright law.

Of course, the layman might argue that without copyright, there would be nothing to prevent others from ripping off content left and right, thus discouraging anyone from publishing in the first place. While this is somewhat true (for there are many who would continue to publish for the sake of creativity), the retort is based on a faulty assumption: That the issue is black or white, that the only choices are the current copyright regime or nothing at all, that there is, nor can be, any in between.

Opinions on Diablo III

Blizzard's latest hit game Diablo III is quite interesting. The first act gives many nods to the original Diablo, from the events of the Fall of Tristram, to the characters involved - King Leoric and the Archbishop Lazarus - even to some of the monsters, such as the Butcher, which could be found in the original Diablo on labyrinth level 2. One of the earliest quests in the game has you saving Deckard Cain, yet again, from demons, in the catacombs underneath the Tristram Cathedral. No, he doesn't reprise his role as your item identifier, as the only items requiring identification are those of rare quality or higher, and you can identify items yourself for free, so long as you perform identification someplace where you won't be disturbed for five seconds. Town portals are the same. 

There's a new lineup of classes, of course, with the only one held over from Diablo II being the Barbarian. There's also the Monk from the original; all classes can be played as either male or female. Your stash, as opposed to being per-character, is now global (transferring items between characters is as simple as putting it in the stash, and then switching over.) So far, I've a Barbarian and a Demon Hunter (roughly equivalent to the Amazon in Diablo II.) Only one class (the Witch Doctor) has Mana; the other classes have class-specific energy reserves which are replenished or charged up in similar ways. At that, there's also no longer Mana Potions. 

Health potions have a cooldown rate - after using one, you're locked from using another for some 30 seconds or so. That's not always a problem, as many monsters drop health orbs, which instantly give you and your mercenary health as they are picked up. Also, health orbs and gold need not be clicked - simply run over either, and your character picks it up. (Imagine how useful that would have been in Diablo II, say, in the Forgotten Tower?) 

There's a few things to be disliked. One, you must register your CD-Key with you Battle.net account, making it pretty much impossible to give the game away afterwards. Two, there is NO offline mode. You must be constantly connected to the Internet to play at all. Adding friends is a two-way process, like on Facebook, and friends can hop into your games at any time - if you want to go solo, they can interrupt and there's not anything you can do about it, aside from leaving the game. The lack of an offline mode is real simple: It's meant to protect the Real Money Auction House, where in-game items can be bought and sold with *gasp*, you guessed it, real money!

The chat system is carried over from World of Warcraft, which isn't all that bad, aside from the help command for chat showing you some chat commands that only work in World of Warcraft. StarCraft II also uses the same chat system, but not as much of it (Diablo III literally has the same floating chatbox, scroll buttons, behaviours, and font as World of Warcraft. Brand identity, I guess.) You can add friends who play StarCraft II or World of Warcraft, and chat with them in-game from Diablo III.

That pretty much summarizes my thoughts. So here's some screenshots of the game, some of which were intended to be used to report bugs to Blizzard. Enjoy!

Note: Click on a screenshot to see it in larger detail.

Getting a Facebook Username

So I was asked recently how to get a Facebook username. A Facebook username is used for several things. First, it's part of your personalized Facebook URL; if your username is johndoe31, then your Facebook URL is http://www.facebook.com/johndoe31. Also, your Facebook username is also your Facebook email account name; as in this example, the email address would be johndoe31@facebook.com, and any mail sent to your Facebook email address shows up in your Facebook messages inbox.

So, let's get down to business.

First, click the downward pointing arrow on the right-hand side of the blue bar at the top of any Facebook page. It will drop down a menu. Select the "Account Settings" option under the menu:

In the General Account Settings page (which should be the first page loaded once you click Account Settings), click the "edit" link on the line for Username (click the image for a larger view):

Afterwards, enter your desired username into the "Username:" box. Please note that you only get to change it once afterwards. When done, click on the "Save Changes" button. Note that the button will be greyed out until you actually enter (or make a change) your username in the "Username:" box. This is to prevent you from losing your chance to change it later by accidentally clicking on Save Changes when you haven't made any changes yet. (Click the image for a larger view):

After that, you should be set! To try it out, go to http://facebook.com/username where username is the Facebook Username that you selected in your account settings.

Some musings for today

I'm sorry, no, censorship is censorship regardless if you're doing it to yourself amidst a "free society" because of "social norms" dictate that one should not speak of certain topics, or if some totalitarian regime is doing it for you forcibly. It's wrong and it needs to end. Period.

And no, there's no such thing as a truly free society. Such a society is a self-contradiction - for it would mean that one person would have to be free to slay anyone they choose, yet that would infringe upon the freedom of his or her victims to live. Not to mention the laws of physics themselves also limit freedom in other, more interesting ways.

---

It was recently published legalization and taxation of marijuana would save 13.7 billion dollars off of the national budget annually. Source.

Yes, I understand it's 0.1% of the total deficit. But it's about common sense (just do some research on what alcohol prohibition did for crime rates and society) and fiscal responsibility. The latter doesn't mean you have to go all gung-ho austerity, but at least find some lunatic fiscal policy and put and end to it!

Oh, consider how much you would save on the budgets of state, county, and local budgets as well. California has a $13 billion dollar deficit for 2012. The national savings in one year alone could put California back into the black with a few hundred million of breathing room on top of it to boot.

Bypassing Gmail's Executable Attachment Restriction

A few hours ago I tried to download a build of a program that was uploaded to uploaded.to, except that with the recent shutdown of MegaUpload, uploaded.to has ceased all dealings with anyone whose IP address originates in the United States of America.

So I had a friend overseas download the program and tried to send it to me, except for the fact that Gmail doesn't allow executable attachments, among others.

So, me and my buddy overseas cooked up an idea to bypass this. What if we just dropped the .exe extension and also used a simple XOR cipher to encrypt the data? Using a key of 0xFF, I came up with this program (after a few bug-fixing revisions, click for larger view):

The source is freely available on the text-snipplet site, Pastebin.com. View the source (updated 2012-01-27 01:32)