Improving FreeNAS iSCSI Performance with VMware ESXi

What does the IOPS setting do?

Included in every version of vSphere is Native Multi Pathing (NMP).  NMP offers the Round Robin Path Selection Policy (PSP) which works by utilising all available active paths to the datastores.  Round Robin sends all IOPS down a single active path until the number of IOPS sent down that path reaches the limit defined by the “IOOperation Limit” setting before it moves on to the next available active path.  Once all active paths have been used then it moves back to the first path and the cycle starts again.

Why change the Round Robin IOPS setting?

By default the Round Robin PSP will send 1000 IOPS down each path before moving on to the next one.  Tweaking this setting correctly can achieve a notable increase in disk IO throughput by more effectively utilizing the available paths.

For those of you that would like to know more, I have provided a link to an interesting article that explores the difference in performance between certain “IOOperation Limit” settings on various P2000 G3 devices here.

How do you change the Round Robin IOOperation Limit setting?

The Round Robin “IOOperation Limit” setting can be managed using VMware vSphere CLI.The commands are slightly different on an ESXi 5.x host compared to an ESXi 4.x host.

Note: A problem was identified whereby the Round Robin IOPS setting would be reset to default on ESXi hosts following a reboot.  This was resolved in ESXi 4.0 U2.


The commands for ESXi 4.x are:

Check the “IOOperation Limit” setting for a datastore:

esxcli nmp roundrobin getconfig -d [device UID]
Example output:
Picture
Set the “IOOperation Limit” setting for a datastore:
esxcli nmp roundrobin setconfig -d [device UID] -t iops -I [IOPS]
Example output:
Picture

The commands for ESXi 5.x are:

Check the “IOOperation Limit” setting for a datastore:

esxcli storage nmp psp roundrobin deviceconfig get -d [device UID]
Example output:
Picture
Set the “IOOperation Limit” setting for a datastore:
esxcli storage nmp psp roundrobin deviceconfig set -d [device UID] -t iops -I [IOPS]
Example output:
Picture

Ideally, you will need to run some tests on your own ESXi environment to establish the best [IOPS] value to use in your specific scenario.  This topic is, however, discussed in another of our blogs as mentioned in the section above.

The [device UID] can be obtained via the vSphere Client by following these steps:

1. Locate the Datastore that you want to set the “IOOperations Limit” setting for.  That can be via the “Summary” tab of a host / cluster or via the “Datastores” Invetory view etc.

2. Right-Click on the datastore and click on “Properties…” in the pop-up menu that apears

Picture
3. Look at the “Extents” section of the datastore properties box that appears and note the naa.600xxxxxx number that is displayed within the brackets.  This is the [device UID] that needs to be input into the esxcli command
Picture

Results

Changing the setting from the default 1,000 IOPS to 1 IOPS yielded the following result in ATTO benchmarks. ATTO isn’t good at benchmarking ZFS storage arrays because of the ARC cache, but you can still see the substantial performance difference here. Whereas before, performance was hitting a wall curiously close to the 1Gb/s saturation limit it now manages to saturate 3 GigE links from the 2048MB test onwards.

100
Performance before set to default 1,000 IOPS
1
Performance after set to 1 IOPS.

 

pfSense 2.2.1 breaks TekSavvy IPv6 on the LAN

So I decided to update to pfSense 2.2.1 today because I had time. Turned out to be a good decision to wait on this upgrade for a time where I wasn’t in a rush – this release changed the way that pfSense handles IPv6 prefix delegation. Long story short, I lost all IPv6 access on LAN clients but had full IPv6 internet access on the WAN. After some searching, I found out that I wasn’t the only one with this issue: https://forum.pfsense.org/index.php?topic=90699.0

Apparently in this case, it’s not a bug, it’s a feature. This was apparently done on purpose according to this post in the pfSense redmine. PD is no longer being requested if you do not have tracking interfaces configured.

The “official” way to make IPv6 work now is apparently to set the LAN interface to Track Interface under Interfaces | LAN | IPv6 Configuration Type. The problem is that you can no longer configure DHCPv6 settings anymore. Apparently DHCPv6 is still enabled but the configuration options are not exposed in the GUI a the moment. It also broke some of my internal LAN due to the static IP address assignments no longer being valid. In the end, my connection was very flaky over IPv6. For some reason, clients were taking a very long time to get their IPv6 addresses (up to 5 minutes). Then some clients started randomly losing their IPv6 internet access again. This piled on top of all the ways this breaks the LAN configuration and internal DNS resolution settings already in place, I decided that configuring it this way is probably going to be unreliable and more trouble than it’s worth, at least with Teksavvy IPv6 addressed handed out via prefix delegation.

In the end, I configured Interfaces | WANv6 | DHCP6 client configuration like this and put everything else back how it was before and it works fine again:

pfsense-2.2.1-ipv6-configuration-teksavvy

Windows 7 USB/DVD Download Tool error: We were unable to copy your files. Please check your USB device and the selected ISO file and try again.

Sometimes if your USB stick has something in its MBR you might get the error

Windows 7 USB/DVD Download Tool error: We were unable to copy your   files. Please check your USB device and the selected ISO file and try again.

You have to start command prompt as an Administrator (On Windows 7 that means right clicking the cmd and selecting Run as Administrator) and use the diskpart utility.

WARNING: Be careful to select the right drive or else your day won’t have a happy end because if you select the wrong drive you will lose all your data on this drive!

Instead of formatting the partition with FAT32, you can also use NTFS, but then you need an extra step to make the drive bootable:

Bootsect.exe /nt60 X:

“X:” is the drive letter of your USB stick. Bootsect.exe can be found on the Windows 7 DVD in the boot folder. However, I can’t really recommend using NTFS. Some USB stick, at least, appeared to be slower with NTFS.

Complete Steps:

  1. Start command prompt as Administrator
  2. type diskpart
  3. type list disk
  4. type select disk  and number of your USB disk ( like select disk 1 )
  5. type clean
  6. type create partition primary
  7. type select partition 1
  8. type active
  9. type format quick fs=fat32
  10. type assign
  11. type exit to exit the diskpart utility
  12. type exit to close command prompt

diskpart[1]

Blocking Ads in Internet Explorer 10

internet-explorer-10-logoInternet Explorer 10 has just been launched and even though it looks visually almost identical to its immediate predecessor IE9, there have been more changes “under the hood” than any other IE update in recent memory.

This version is faster, follows standards much better and dare I say actually usable. One major feature that many may miss when using IE as opposed to either Chrome or Firefox may be the ability to easily block ads with browser extensions such as AdBlock Plus.

The good news is that Microsoft has now integrated “Tracking Protection Lists” (TPLs) straight into the browser. In effect, this allows you to add block lists straight into IE. This functionality can be used in effectively the same way as AdBlock Plus in other browsers.

Blocking Ads with a TPL

To block Ads using the TPL feature of IE10 is very simple. Just visit this page: http://www.iegallery.com/en-us/trackingprotectionlists and add the relevant list to your browser. The list you’ll want to select here is “EasyList Standard” which is exactly the same list that AdBlock Plus uses. Once you click on “Add” next to that list, you will be presented with the following dialog:

adding-tpl-ie10

Here, click on “Add List” and it will immediately become active.

Disabling the TPL

If you ever want to disable or remove the list, use the following procedure:Editing TPL in IE10

  1. Click on the gear icon in the browser
  2. Hover over Safety
  3. Click on “Tracking Protection…
  4. Left click on the list you want to edit
  5. Click on either Remove or Disable

 

That’s pretty much all there is to it! Hopefully Tracking Protection Lists will help make your experience in IE10 a little bit better. What do you think of IE10 so far? Don’t hesitate to leave comments below and let us know!

Why Tablets will Never Replace Laptops

Tablets PCs are all the rage nowadays. I mean, lets face it – the personal computer is no longer the king of technology. It’s been around for decades and it’s high time that we Tablets on a Shelftalk about something fresh and new. Smartphones are of course still hip, but the release of Apple’s original iPad in April of 2010 has shaken up the market with a completely new viable form factor for the media to go crazy over.

The entire market has also been presented with a device which is genuinely more portable, generally offers better battery life than laptops and is far more intuitive with multi-touch capabilities. The experience is simply far more interactive. Tablets certainly have their place, but still have their pundits. Some are still left asking themselves why they need yet another device for browsing the web, viewing pictures or writing emails.

There is far more to it than that, however. Here are the top 5 reasons why tablets will never be able to completely replace laptops:

  1. Nobody would ever want to write this article on a tablet. I mean, seriously guys, think about this one. By the time somebody typing the 1,000 or so words (hopefully less!) it will take to get my thoughts across in this article, a writer equipped with a laptop could easily write 2 or 3 full articles. Then there is email, programming, Excel, databases… I think you get the idea: the keyboard is simply the ultimate input method for writing… anything. Period.
  2. Just because you can make it smaller, doesn’t mean you should. This point goes hand in hand with the above point. We all like small and portable things. But the smartphone revolution has made this point abundantly clear as well. When we found out how powerful that ultra-portable technology like smartphones could be, we also began to recognize that a bigger screen could also be a huge asset which would allow us to work with more data at once. The keyboard and mouse have a similar relationship with the tablet as the tablet does with the smartphone. After all, we are human and we will always need devices that are large enough to interact with. If our eyes could resolve the hairs on an ant we could certainly live with much smaller screens. If our hands were smaller, we could certainly live with smaller keyboards. There is a finite limit to how small something can possibly depending on how we choose to interact with it.
  3. Higher-end content productivity is simply better served with a full-fledged system. I’m talking photo editing, video editing, CAD drawings and the likes. Not to mention that full-fledged laptops are more powerful (though in 10 years this may not even matter anymore). But the venerable mouse and keyboard with various other input devices attached have certainly proven themselves. In fact these are environments where all the input devices required almost favor having a fixed workstation. Let us also not forget more advanced multi-display setups as well. These are largely fixed and take advantage of features which tablets have no place offering.
  4. Not everybody is always on the go. Really, I promise! Everybody is increasingly mobile nowadays? Tech support representatives are more mobile? Accountants are more mobile? Not everybody is more mobile, and that’s fine. But somebody has to call out this fallacy that absolutely everybody wants to or even can work while driving or flying to some other destination. And besides, wasn’t technology supposed to help people stay put and avoid having to fly out to meet your client in the first place? What happened to green initiatives focusing on reducing how much you travel? When did fuel get cheap again? I must have missed those memos.
  5. Tablets are better suited to content consumption than creation. That really seems to be the recurring theme here. This is not derogatory. This does not mean that tablets can only be used to view YouTube videos of people getting kicked in the groin and pictures of lolcats. The CEO of a company is more likely to be a consumer of content than a creator. The engineers, marketers, accountants and lawyers create the content, among other groups of skilled workers. Managers will then look at all that information and make better decisions based on it. But at the core of everything, there is always a huge need for creation of new information and content. Tablets, mainly due to the different interface, are better suited to organizing and displaying the information once it’s been created.

So given all this, what can we deduce? First, tablets are complementary to laptops in the same way that smartphones are complementary to them. This often seems to get lost in a lot of the media coverage surrounding them.

Further, to those who do not understand what purpose a tablet serves in an office environment, I’d be willing to bet that you don’t spend much time reviewing lots of information while on the run, you don’t work with information often, or you are the one creating it.

So no, I can’t predict the death of the laptop even within the next 10 years anymore than I can predict the death of the full-size keyboard in that time frame, and nor should you. It’s absurd to even think about it when you put it that way and it always will be as long as humans have 10 fingers and two hands.

Apple Loses Another Unreleased iPhone… Or Not

Hot Girl Holding iPhone
Hint: she doesn't have it.

As you  probably remember  last year, an Apple employee who was testing the iPhone 4 prototype in the wild lost the prototype in a bar. The man who found the phone quickly realized what it was and contacted well-known tech blog Gizmodo and sold it to them for $5,000. Apple was apparently far from pleased with the incident as they started a police investigation and tried to sue the pants off Gizmodo, which subsequently settled. In a thoroughly altruistic mood, Apple also ended up firing the employee that lost the phone despite a large public outcry not to, which included Apple co-founder Steve Wozniak.

Fast forward a year, and we have found that CNET is reporting in an apparently “exclusive” that the latest iPhone 5 prototype has also suffered the same fate… in yet another bar. When I heard that a couple of days ago, I thought it sounded a little fishy and promptly moved on. Well it turns out that an editor at SF Weekly called the San Francisco police. Turns out they had no record of any such investigation – exactly what they told CNET.

Apple has not responded to this incident publicly, yet.

The Truth?

In this case, something doesn’t add up and clearly somebody is lying. Consider all the possibilities:

1. Apple intentionally lost this phone to steer attention away from the recent departure of CEO and co-founder Steve Jobs.

2. A prototype was accidentally lost, but the police don’t know about it or are lying about it for some reason.

3. CNET fabricated this whole story to get all the press coverage. They get an exclusive high-profile story without having to shell out 5 grand as Gizmodo did.

I’ll let you decide, but personally I have a sneaking suspicion that this is a CNET fabrication, and a poor one at that. I can’t possibly imagine Apple employees are so careless that they would be running around with iPhone prototypes and losing them in bars. Especially just a year later. What’s more, Apple really didn’t seem to be too pleased with the loss of their phone last year – and they certainly weren’t shy about making that crystal clear.

Then we come to CNET, they have the most to gain from this by far. This story is just too perfect and they have too much to gain.

Seemingly Magical Seamless Computing from nsquared

nsquared Seamless Computing Microsoft SurfaceIt seems that technology is breaking down the boundaries of what is science-fiction and what is real yet again. A company called nsquared Solutions has put together a series of Microsoft technologies that put some Star Trek tech to shame.

Largely, this seamless computing consists of completely wireless communication between devices such as a Windows 7 mobile phone, the Microsoft Surface, a Windows 7 tablet and Kinect. The large breakthrough is how these devices seamlessly interact with each other and complement each other. Gone are the days of the antiquated progress bar.

[youtube_video id=”oALIuVb0NJ4″]

In the following demonstration, Dr. Neil Roodyn receives an email on his phone and then reads the data from the Surface. He then pulls up architectural plans, overlays a different view on the tablet and then navigates the 3D model of the house using a TV powered by Kinect. The computer, called “Bill” in this video responds to voice commands and helps him along in his architectural madness while he scans in 2D pictures he just took into 3D objects that get thrown into the mode.

This technology all comes together almost magically. Now if only things technology was this seamless to use everyday!

Fix Windows Backup Error: 0x8078002A on 2.5TB and 3TB WD Drives

“One of the backup files could not be created. Detailed Error: The request could not be performed because of an I/O device error”

Hard DriveLook familiar? Have a Western Digital hard drive that you are trying to back up to?

I would like to quickly mention a solution that I found for larger (2.5TB or more) Western Digital hard drives that use “advanced format” 4KB physical sector sizes. Many users complain about an issue with Windows 7 Backup and Windows Server backup which spits out the useless Error 0x8078002A.

This worked for my 3TB WD Elements drive. Out of the box this drive did not work with Windows backup at all. WD support actually came through for me on this issue and pointed me to the following solution:

http://wdc.custhelp.com/app/answers/detail/a_id/6618/session/L3RpbWUvMTMxMzUzNzc5My9zaWQvT1hrMSpJQms%3D

Basically it consists of formatting the drive with their external drive formatting utility in “Factory Default” mode rather than XP-Compatible mode which (ironically) is how the drive ships. After performing the format, the drive works fine with Windows backup in both Windows 7 and Windows Server, but is incompatible with Windows XP. In my case, this is an acceptable compromise.

This is how the drive appeared out of the box:

C:Windowssystem32>fsutil fsinfo ntfsinfo H:
Bytes Per Sector  :               4096
Bytes Per Physical Sector :       <Not Supported>
Bytes Per Cluster :               4096
Bytes Per FileRecord Segment    : 4096

This is how the drive appeared after the “Factory Default” format:

C:Windowssystem32>fsutil fsinfo ntfsinfo H:
Bytes Per Sector  :               512
Bytes Per Physical Sector :       <Not Supported>
Bytes Per Cluster :               4096
Bytes Per FileRecord Segment    : 1024

Finally, I should point out that this utility will probably not work for drives from manufacturers other than Western Digital as it scans for WD drives specifically on launch and is likely working directly with their drive’s firmware. That said, I do believe this should cover all WD drives 2.5TB and larger to correct this issue.

Writing Regular Expressions for .htaccess and IIS 7 URL Rewrites

Regular ExpressionsWhen I was in the throws of transferring this site from Joomla to WordPress, one issue that I had to contend with was the URL changes. The solution is the 301 redirect. That flows link juice from the old URL to the new one and if possible to implement is far more useful for users.

This would be a huge pain to implement manually for every URL and thus had to be systematic. The easiest way to accomplish this is through the use of regular expressions in what is known as a .htaccess file on Linux/Apache or Microsoft’s IIS 7 equivalent functionality known as the URL Rewrite module.

To use this guide, you’ll need to know the basics of setting up rewrite rules. Turns out that the regex not that hard though. In fact, a simple chart can describe all the behaviors that regular expressions follow.

Values:

All the values you will find in this section will be matching against a single character.

. Period matches any single character including numbers, letters and symbols.
ex.:
A, a, 1, %, etc.

[ ] Square brackets will match against a range of characters.
ex.:
[a-z] matches all lowercase letters
[0-9] matches all numbers
[abC] matches a, b or C
[a-f0-3z] matches a to f, 0 to 3 and z

[^] Matches against anything not in brackets. Basically a negation.
ex.: [^a-c0] matches any character that is not a to c or 0.

Anchors

All the values you will find in this section will be matching against a single character.

^ Means the string must start here
ex.:
cats will match “I like cats”
^cats will not match “I like cats”
^cats will match “cats like me”

$ Means the string must end here.
ex.:
cats$ will match “I like cats”
cats$ will not match “cats like me”
^cats$ will match only “cats”

Quantifiers

All the values you will find in this section will be matching against a many characters. You should always use the most specific one possible to reduce overhead and false positives.

* matches zero or more of the previous character.
ex.: lo*l will match “ll” “lol” “lool” “loool” and so on.

+ matches one or more of the previous expression
ex.: lo+l will match “lol” “lool” loool” and so on.

? matches zero or one of the previous character
ex.: pi?e will match “pine” and “pie”

Others

| means OR.
ex.: “gray|grey” will match “gray or “grey”

() will group an expression together and allow it to be reused later (more on this later)
ex.:
The example above could be simplified to “gr(e|a)y”.

Escape Character

Since regex use special characters as defined above, we need a way to tell the computer to read the character literally. This is done with a backslash .
ex.: “.htm” must be escaped as “.htm” to treat the period literally.

Example in Practice

So lets put all this together and put it to good use. We’ll use the case of this site as an example. The first thing to do is observe the structure of what you have and see which strings can be easily grouped together in addition to which parts of the URL you want to keep and which you don’t. Take this for instance:

Old URL
www.cmoullas.net/reviews/34-mobile/46-latitudereview

New URL
www.cmoullas.net/latitudereview

We can see a few things here. First, the end of the URL will be staying the same “latitudereview”. That means we want to isolate the end of the URL. We also want to be as specific as possible to avoid false positives and avoid overhead. Finally, we know that we only need to match after the first slash and don’t need to worry about the www.cmoullas.net/. So consider the following solution:

[a-z]+/ matches “reviews/”
[0-9]+- matches “34-”
[a-z]+/ matches “mobile/”
[0-9]+- matches “46-”
(.+) matches “latitudereview” and saves it for later referencing

Putting that all together:

[a-z]+/[0-9]+-[a-z]+/[0-9]+-(.+)

This will great, but we can still improve on it. How you might ask? Well remember in this case our only goal is to get at the string “latitudereview”. As a result, we actually only need to match “/34-mobile/46-latitudereview” as that will be sufficiently unique to not create any false positives. Thus we can get away with just:

[0-9]+-[a-z]+/[0-9]+-(.+)

A couple more notes on this and you can call yourself an expert! Notice the specificity here. We matched as follows:

Any numbers and a dash
Any letters and a forward slash
Any numbers and a dash
Any string and saved it

We can do this because we know that the URL will always follow that pattern. At the end, we had to match any string because the title could be far more complex with any letters, numbers or even symbols, such as latitude-review-2, for example. The rest will always follow the same specific pattern.

And that’s it! If you followed all this, you’ll be following all the best practice rules with regex and will be rewriting like an expert! Look forward to another article in the new future explaining how to use back-references in both mod_rewrite .htaccess files as well as in IIS!

The Truth About Microsoft

Apple vs. MicrosoftI’m continually amazed by news columnists that predict the death of Microsoft. Just this morning I was reading the news and saw a slew of articles in Google News about predicting a death within the next decade and possibly within just five years from now. So many people seem to hear the knell of the death bells. So are a bunch of people really hearing things that aren’t there? Yep. Here’s what they are hearing, why they are hearing it and why it’s all in their imaginations.

Usually the main point that is brought up is that Microsoft has no idea about what Web 2.0 is. After all, their browser market share has been taking a beating for years now and is now below 60% from near total domination. This comes amid allegations from Google that the browser is the way of the future and the OS won’t matter sooner rather than later. I wrote a very popular, if not controversial column on why Chrome OS will fail.

Regardless, Microsoft won’t dominate the browser market, that much is for certain. But there are other markets to consider. The phone market. Okay, well they lost that too. Clearly they don’t stand a chance then.

You also hear people cry foul about the fact that Microsoft has dumped Windows Live Journal (their mostly unheard of blogging platform) in favor of collaborating with WordPres to provide a better blogging experience for their end users.

Tablets? Microsoft has been pushing them for years without much success. Then Apple comes along and shows everybody a radical new way of doing it which has becoming a huge success.

Wow, that really does sound grim, doesn’t it? Those bells are ringing louder than ever now… But no matter, it’s all in your mind for one simple reason: most people have absolutely no understanding of what Microsoft actually does. Lets start with that and then go on to explain why Microsoft is actually stronger than ever (not to mention that their stock is soaring for the Q3 2010 and analysts are in full agreement). No, we aren’t all nuts. Here is the real story.

First of all, all that the average person off the street sees from Microsoft is Windows and Office on their home computer. They probably also have a work computer with a Microsoft installed whether it be 2000, XP, probably not Vista or 7. You might ask yourself  a question that rarely gets adequately answered: why do businesses run with Windows all the time? It’s horrendous. Then some people will say that they simply have old applications that were built for Windows which would be a pain to port over to something better or newer. Maybe, but that’s a very myopic view of the whole situation.

The reason that the Windows OS is run in the corporate world is simple: domain controllers equipped with Active Directory and desktops that fall in line with whatever the domain controller says. Say what? Active directory is one of the many things that most people don’t know about that is a prime example of how little most realize is going on. It is quite simply a way to manage groups of computers centrally from a server. You can remotely turn a computer joined to a domain to anything from a kiosk that runs one application and is totally locked down to a full development machine. You can enforce corporate policy and create a uniform and managed experience for all your users. The truth of the matter is, companies absolutely love it, and it really is all that.

Windows is actually built from the ground up to be run in a managed environment. The version that most people run at home has almost always traditionally been a port from the corporate built that allows users to run standalone machines. It was always an afterthought though, and the focus always has been on business requirements.