Intel® Software Network Blogs » Kevin Pirkl (Intel)

JavaScript Paths,Querystrings and Encoding

Kevin Pirkl (Intel) — Wed, 02 Jul 2008 16:17:22 +0000

Rick Strahl's "Making sense of ASP.Net Paths" has been inspiring to me since I have visited the page quite often so I thought I might do a quick similar post for Client Side JavaScript code and some How To's with some AJAX know-how sprinkled in.

Firefox and Firebug add-on developer tool make undestanding JavaScript's document.location much easier.

Here at the Intel Software Network (ISN) we have quite a few AJAX based services (Feedback, Email A Friend, Login/Logout, Polling, Rating, Commenting) that can be dropped into any and all of our 40-50 web sites by placement HTML elements with specific ID's. Each common service is also written to support localization. You can check out some of these functions in the button bar of the following pages US, China and Russia ISN Landing Pages. Just load the pages and look for the icon bar at the top right of the page.

Most of thes services require that we pass the URL back to the server to gets back data. When passing a URL as a querystring parameter encoding it correctly is important to prevent unwanted side effects. For instance some of our fist passes at adding a Comment Service to pages resulted in differnet sets of comments getting returned when the querystring varied.

So lets take a peek into the Firebug DOM tab and see what document.location brings.

As you can see everything is nice and easy to get at here.

Here is some sample HTML code that you can save and run on your own local machine and see each of these variables at work. Try adding QueryString parameters (and/or even add a full properly encoded URL on the end.) My most complex resulting output was http://localhost/locationtest.htm?a=1&b=2&target=http%3A%2F%2Flocalhost%2Flocationtest.htm%3Fa%3D1%26b%3D2

[


	Document Location and encodeURIComponent()
	




Tools - Firefox and Firebug

document.location = 
document.location.href = 
document.protocol = 
document.location.host = 
document.location.port = 
document.location.pathname = 
document.location.search = 
document.location.hash = 

This is not the base URL for this page.
var notBaseURL = "";

The base URL is -
var thisURL = "";

The base URL plus parameters -
var thisFull = "";

Say you want to call an AJAX service and you want to pass this URL correctly.
I would consider that for a service call in most instance like getting Tags for this page
Or getting Comments for this page you would only way the base URL.
"";

To make an HTML call and pass another URL you need to use the JavaScript method
encodeURIComponent() = "";

So your final call to your service URL for passing thisURL would be something like this example.


Or with your querystring parameters it would be like so.


Dont include document.location.hash in your call to your service as it can cause issues down the road.





]

Here is the output from the above HTML

It might seem very simple and you might say why bother but the end result are cool. Take for example out Coment Service. For this we need only the base url and not all the extra parameters. If you were to take the extra search parameters for Comment retrieval then you would get one set of results and without the search parameters another set of data. All our Content Deliver pages allow for Comments, take A Library Based Approach to Threading for Performance for example (drive to the bottom of the page to see the add a comment option.) The resulting call that populates the return data set comes from this page call (post page load)

http://softwarecommunity.intel.com/isn/home/CommonServices.ashx?F=GetCommentsJSON&P1=http%3A%2F%2Fsoftwarecommunity.intel.com%2Farticles%2Feng%2F3816.htm&P2=&_=1215014917424

The packaged JSON data set uses JSONP for the callback pattern and thus to populate the page data.

Simple but very cool!

Cheers

Kevin Pirkl

Google IO Conference - Day 2

Kevin Pirkl (Intel) — Thu, 29 May 2008 23:45:45 +0000

From day 1 registration - Oh the panic in the crowd at Google IO 2008 #io2008

Summize #io2008
Twemes #io2008

I'm going to let this one ride all the way as it (I'm not big on spelling corrections, etc..) I've typed enough notes today and I'll have to take another look at it tomorrow and perhaps then make additions and/or corrections.

Enjoy, KP

Day 2 begins with Keynote speaker Marissa Mayer, Vice President of Search and User Experience talking about "Imagination, Immediacy and Innovation... plus a little glimpse under the hood at Google"

Blah, blah, blah, laughter, blah, blah blah nothing we have not heard before for those that have seen the TV documentary.

Here's a cool fact I noticed.. The main stage has the Google IO logo and the left and right are top and bottom like so

            1  1111

            1  1  1

01001001    1  1  1   01001001

01001111    1  1111   01001111

The bottom left and right are displayed as black dots and white dots in a set of eight or Binary to ASCII that is "IO" which is a nice little play on the whole Google IO thing. (??Hint: Times Squaring the Circle???)

Ok back to the keynote: This story Marissa is telling is the one about Google Home Page A/B testing and returning more than 20 results with the end result being "It's not about the number of results but it's about the speed at which those results are delivered." Nothing that I didn't already know. What crazes me about this is that they cite a falloff based on speed when it is a know fact that user percentages that go beyond page two of a search fall off considerably. With today's faster net speeds there is no way you will convince me that this is not about advertising $$ based on this logic.

I didn't know about the Swedish Chef Easter Egg. (Egg's and Hoaxes)

Note: There are more laptops here that I have ever seen before at a conference. It looks like 90% of the people here are wired in.

Note2: The provided WiFi is not keeping up with demand so I am glad I am putting these notes together in Windows Live Writer.

Note3: The told us that all the sessions will be online next week (I guess that means videos will be online.)

Note 4: Open Social API gadgets.util.sanitizeHtml() for HTML cleanup and protection. Funny thing is that you can find this in the version .8 release notes but not in the API reference.

Note 5: Need power strips everywhere considering the number of laptops here. The future is having a good laptop and conferences will continue to have more and more laptop percentages.

Note 6: I hear there are around/over 3000 persons here. I think 2/3rds of the group are using Mac's and the rest a mix of PC's, Mids,iPhones, other mobile types.

Note 7: I need to get a sitemap so sites like Google can better find out content.

Note 8: Images from Google IO 2008 (tagged with the io2008 tag)

Note 9: I keep seeing something called hi5 mentioned with this logo

and along with that and a few other OpenSocial containers.

Note 10: Tons of people taking pictures of slides in lectures and even saw a few webcams recording or broadcasting perhaps... Hmm I wonder if I would blue-screen my T61 if I try a Yahoo Broadcast. I better save this document first.

For my session lineup today:

Session 1: I picked "HTML5, Brought to You by Gears" - Speaker Aaron Boodman - (Room 1)

One other interesting note is that I think most of yesterday's sessions I picked were in smaller rooms that did not fill. This session is in a double size room and packed tight onto the floors as well.

Gears and HTML5 web standards - Gears is incrementally implementing HTML5 standards as they release new and exciting features. They consider themselves ahead of the curve on this because they are doing things that are being set as standards later on. This means they have to come back and the adhere to the accepted approach that gets put forts by bodies like the W3C.

Gears core functions are is typically wrapped in their own JavaScript Namespaces so they can later fix the underpinnings to be HTML5 compliant.

The DB demo that they were going to show didn't work which was kind of annoying IMHO. Ok with some help from the crowd they did get it finally working. Basically it was a demo of native implementation of the new HTML5 DB API using Safari 3.n and then switched to using Gears which in essence makes it backwards compatible with older browsers not yet supporting HTML5 DB.

Session 2: OpenSocial, OpenID, and OAuth: Oh, My! (Room 4) - Joseph Smarr -

I was going to go to "A World Beyond AJAX: Accessing Google's API's from Flash and other Non-JavaScript Environments - Speaker Vadim Spivak - (Room 9)" but I think those are things I could learn for myself so I picked the OpenSocial,OpenId track in hope of getting a future Intel integration up and running.

OpenID enables one place storage of Personal Profile information and allowing sharing that information with websites that you trust. (Note: I don't honestly understand what this speaker is suggesting here using the Google offerings.)

My interest in the OpenSocial side of this talk is honestly nill and I am more interested in knowing more about the OpenID and OAuth integration points.

Joseph mentions that all social apps are broken or act like silos. I tend to agree with that. OpenID would be a good thing for Intel.com to work out. Hell for that matter working with Google Auth would make Intel.com about 1000x better.

Crumb I think this whole Social web talk was an excuse to show us Plaxo (From their site Plaxo is about the be bought by Comcast.) I have heard all I care to hear about OpenSocial and what it is and I thought this talk was going to be about OpenID integration process steps etc.. When the frick is Joseph going to dive into OpenID and OAuth?

I'm personally not very excited about ever allowing third parties to access my online address book or personal details beyond the information I wish to give. The thought putting all my interconnections in one place thinks that it creates a ripe place for thievery or scam.

Finally a two second mention about XRDS but so far that was it.

This was not really a session for developers.

Lunch

Session 3: Inside the Android Application Framework
(Room 3) - Android Engineering Team -

APK Runs on a single process

Activities - Concrete API

Task - are a Runtime Sequence of Activities - Tasks can span Activities

Process Basics - One per APK - By default one thread per process and all APK run within that thread. Process is started for a given UserId

Activities Lifecycle -

Starting Up (Created(), onStart(), onRestart(),onResume())
Normal Execution (onFreeze(),onPause())
Shutdown (onStop(),onDestroy())

Example Lifecycle -When a child Activity is launched

Live sucker...

Die sucker

Threads and Loops - Loopers Assume they are associated with a single process and are not intended to be thread safe. Loopers cannot tolerate multi-threaded access and must use message passing/handlers instead.

The thread handling was quite interesting

Process Resource Management

Only three process run as root

init
Zygote
main runtime

To connect distinct process's, for that process you have explicitly expose it via Services API and Inter-process communication bridges.

Dalvik VM - The Virtual Machine that hosts a running Android Application instance running in a single process.

Session (Long break): I plan to take a break now instead of a session to see all the offerings on the main display floor.. I will tell what I see and what piqued me here later...

Google Health API - Google Health TOS gets around normal HIPAA requirements (If you create, transmit, or display health or other information while using Google Health, you may provide only information that you own or have the right to use. When you provide your information through Google Health, you give Google a license to use and distribute it in connection with Google Health and other Google services. However, Google may only use health information you provide as permitted by the Google Health Privacy Policy, your Sharing Authorization, and applicable law. Google is not a "covered entity" under the Health Insurance Portability and Accountability Act of 1996 and the regulations promulgated thereunder ("HIPAA"). As a result, HIPAA does not apply to the transmission of health information by Google to any third party.)

Google AJAX Libraries API - Provides hosting and loader for various JavaScript Libraries and a loader for my favorite jQuery -

I met Vadim Spivak the person who put together the hosting of all these various OpenSource JavaScript libraries and gained a few insights into the loader mechanisms. You can now reference your favorite JS library directly from Google CDN but for even better loading speed Vadim recommends you use the .load() API in the HEAD section with explicit versioning (actually does a document.write) which will get you the file from cache servers much faster than direct linking. Cool detail to know.

Session: Creating a Client Side Search Engine with Gears - Brad Neuberg -

What would be very cool to get out of this session would be the ability to take our Intel CD system (internally called Redfort at Intel) offline for and make it searchable. Imagine thousands of Intel.com technical documents at your fingertips for easy reference.

Demo was awesome.. Brad showed a web app that loaded a text document containing URI's of E-Books. This app parsed and pulled all those E-Books local and then provided a search interface that used FTS2 (Full Text Search2) Gears API function to search the entire database and return results. Wow!!!

Sample used Gears threads to do all the work making the app run without stuttering. Brad's explanations were excellent and he showed a lot of code which was very cool. I'm not a Dojo fan but between Dojo,Google style code and jQuery, I think I will learn more about those first two. Different style and approaches teach a person more than just knowing one style of coding.

The project is not available for download as it is not completed but I did find a Google reference that led me to gears-pubtools (confirmed, it will be there when it is completed.)

Kick Azz demo for sure and I will look forward to see it.
Kevin Pirkl

Google IO Day 1 - Twitter #io2008

Kevin Pirkl (Intel) — Thu, 29 May 2008 03:46:46 +0000

If your curious about what's going on at Google IO conference you can follow the converstation via the following two links.

http://summize.com/search?q=%23io2008
http://twemes.com/io2008

I'm sure there are more ways and other tags people are using but the two above are simple to follow.
....

Note: I wont vouch for 100% accuracy on the details below and much of this is just notes but here goes anyway...

Well I'm here at the Google IO Conference 2008 and the check-in lines are long and they are getting a late start due to very long lines still waiting to get checked in. (10 Minutes late so far.)

Quick Thoughts: 15 minutes late now, this place is packed, great round style layout for the keynote address, they finally caved in and just let everyone upstairs (without badges or checking if they have paid.)

They are starting the event now...

The sessions lineup looks great for me today..

Keynote: Client, Connectivity, and the Cloud - Vic Gundotra, Vice President, Engineering
Extend the Reach of your Google Apps Environment with Google APIs
(Room 9)
Lunch
Rapid Development with Python, Django and Google App Engine (Room 1) (Skipped this for Building an Android Application 101 (Room 3))
Can We Get There From Here?
(Room 4)
Secure Collaboration - How Web Applications can Share and Still Be Paranoid
(Room 4)
Authenticating to Google Data Services
(Room 9)

Keynote - Google/Other announcements so far..

MySpace Mail Message center goes Mobile with announced Gears support.

Google App Engine (GAE) announces two new API additions, the ever popular MEMCACHE and a Image Manipulation API. GAE Opens up the doors from limited beta by waiting list invite to all are welcome. They also announced their pricing plan.

Cool Item: Keynote showed off an AJAX API demo (we have all seen it before) but what was cool is that they were using the not yet finished jQuery 1.2.6 release. Google's commitment to Open Source is awesome. Google now provides hosting for many different Open Souce AJAX JavaScript API's so you no longer have to maintain your own versions/versioning of jQuery because Google does it for you. AJAX Libraries API

GWT 1.5 Release available today with JAVA 5 Language/Feature support.

Session: Extend the Reach of your Google Apps Environment with Google APIs - Lots more JSON-Script style support for web apps. I liked the information for .Net about the Google Document List Data API that allows you upload multiple DOCS into Google documents. SalesForce.com did not interest me much.. Wish I had gone to the AJAX track but was good information.

Session: Building an Android Application 101 (Room 3) -

IntentRecievers are listeners designed to respond to broadcasts. They are like a verb and object; a description of what you want done or an action (VIEW, PLAY.) The core System matches Intent with actions. One cool thing about Android is the Integrated Browser called WebKit which is an open source project. This API set is so cool I think that I might drop my iPhone just to write apps for it. Well perhaps not but I could get another phone. Wow the whole hour is already gone..

Retweeting @gmitsopoulos: #IO2008 Android is scheduled to ship on first handhelds in second half of this year

My earlier session ended early so skipped over to another session and managed to get in a question as they were talking about the Google Language Translation API. I fired off a question about contextual HTML parsing and got back no real answer. Post event one of the API team members approached and told me he had read some of my posts/reviews on the Translator API over the last year. (Very cool to know that I'm being read, wowza.) They did mention some improvements in the API with HTML handling which means I should take another look at it again.

Session: Can we get there from here? - This is a tough topic for the speaker to take on. They had to delve quite a bit into overview and historical tense to get to the root of where things are going (looking at user needs are now and anticipating what they will need later.) The speakers position seemed to be that the worst browser sets the bar that we get stuck with regarding HTML rendered pages. Speaker said what is needed are better tools that will allow for ouicker delivery of newer browsers versions supporting the latest "accepted" standards. Alex Russell (the speaker) mentioned that he thinks that "Browser Sucks" (but he can clarify his position here if I got it wrong.) So many browsers that just don't work together, provide standard API's, standard formatting on platforms from desktop to mobile phones, etc. His opinion of what will make these browser vendors better is more competition. One thing that is true though is that we are seeing more major upgrades much faster in recent years and new major versions for all the major browsers as (perhaps) due before the end of the year.

Mono-cultures reduce costs in the short run - (Not sure what that exactly means but I wrote it down.)

There exists an idea that web developers are hostage to deployed web browsers and the realm of web-ish applications although cool do not get us to the desktop-ish applications that Silverlight and FX take us too.

BTW - Alex Russell is a frigging genius and I need to get on board with his preaching's.

Session: Secure Collaboration - How Web Applications can Share and Still Be Paranoid
(Room 4) - Mike Samuel

Douglas Crockford's ADSafe and Google Code Caja Project are just to approaches to securing third party code.

Turns out that Caja is not yet usable today but there are prototypes. Not much use to me though since this is just all theory.

Was very short presentation..

Break - Sitting around attempting to digest the new and old. It's amazing how far things have moved since Google Dev Days about a year plus ago. Software changes are so much more rapid these days and that is for sure one thing to look for in my own work and designs. I think foundations for rapid change is a better sought after approach than doing things the manual way.

Session: Authenticating to Google Data Services
(Room 9) - Google Data API Authentication Overview

Google Data API's about

ClientLogin - Stored on the client machine only.

When you cant always do a browser redirection. Typically uses HTTPS and sometimes CAPTCHA when negotiating security authorization.

HTTP POST URL - CGI Headers (accountType,Email,password,service,source) and gets back plaintext key-value pairs SID LSID and Auth. Auth is the important token that you use in the authorization header to get a users data for them.

AuthSub - redirection style login using redirection.

Redirects to a Google branded web page and gives approval for your web application and redirects the user back to the originating page. Apps have to be registered with Google for this to work. Remember that this is all about Google user personal data access.

Scope - New Is a new feature that allows coverage over many different user Google Data sets (Calendar,Email,Contacts,etc..)

This is a more complex approach and to protect against this the web app domain must be registered with Google in order to use AuthSub. This approach uses Asymmetric Cryptography with Public/Private keys for verification. You own your Private Key signature and it's a PKI like exchange.

myrsakey.pem - private RSA key

myrsacert.pem - public RSA certificate

opensll req -x509 -nodes -days etc...

1> Register your account domain with Google (for example softwarecommunity.intel.com. They use an HTML file you place into the root of the server.

2> Then you Agree to a TOS and upload your certificate till it becomes active.

When complete your ready to roll

Client Libraries are available in Java,.Net,Python,PHP,Objective-C

OAuth - net step for AuthSub just more complex.

Open standard with specification still being developed with Open Source Libraries already available.

Most of this stuff I already know but what the heck it was very good to get the juices flowing to see how I can better our SoftwareCommunity.intel.com offerings down the road.

End Day 1 - Time for dinner and a nap...

Kevin Pirkl

Google AJAX Language API first impressions

Kevin Pirkl (Intel) — Fri, 21 Mar 2008 20:24:28 +0000

Darn I have been one-upped by Google and they finally implemented a serviceable API onto their language translation services!

Google announced yesterday a new AJAX Language API and while language detection is not high on my priorities the actual “contextual translation” of HTML and TEXT has always been of great interest to me. In fact I create a screen scraper version of Google HTML Translator page and wrapped it into a nice jQuery JavaScript library for fun.

New Google AJAX Language API - Tools for translation and language detection

My original work (out I might mention it was done over a year ago) was finally posted to my personal blog ZombieBob's Weblog on March 11 with the full article on Google Docs “Spinning SOA onto Google Language Translator” shows off some original work creating a translator API that screen scrapes the Google language Translator page.

You might want to peek at the examples page that contains my original work on the subject quite a few implementation samples Localizing Web Page Elements using jQuery

Here is the documentation for the Google AJAX Language API and the Class reference can be found there as well.

My work started from this Google example translator page saved to disk and modified a bit to support a little more text and jQuery style code and here are some of my notes.

I started with a TEXTAREA field so I could submit more data at one time and stuffed in the following HTML for test purposes.

This is an test article for DPD. This
article

belong sto Multi Core.

Section I - This is good stuff

This is the stuff that I like to see..

Article Heading A

Subsection - I.1

Browser imposed HTTP GET limitations

Around 2048 characters or 2000 characters for the path portion. I Googled the terms “Browser imposed HTTP GET limitations of 2048 characters ” for you and came up with this “WWW FAQs: What is the maximum length of a URL? ” which is good enough to explain some details the limitation.I stuffed the test page with the HTML above multiple times so the number of characters would go well beyond the HTTP GET limit of most browsers so I could show a point. In this case you have an character limit and imposed cross domain security constraint if prevents and API (and this one as well) from doing a POST and thus getting around the 2048 characters limit. The end result is that the API will bomb because Google’s API doesn’t check the URL string length for this limit and warn you about it. Perhaps they should be checking this. Note that this just means that you have an imposed limit to the size of the text that your planning on submitting to this API and you should plan and code accordingly.
Context - First letter capitalized

Note that after a period the return translator text capitalizes the first character in the DIV tag.

DIV is moved - Single word context nested in HTML

When a word is encapsulated within an HTML tag it gets moved. Notice that the word "article" on the return translation gets left shifted. If you put in an HTML STRONG tag you can see the same results. This is pretty annoying.

Carriage Return/LineFeed and HTML Entities translation resultsHere is a another test to understand some more about data processing glitches. Data submitted was the following hello\r\nworld

I used Microsoft Fiddler to capture the following underpinnings

Microsoft Fiddler copy of the captured URL is

http://www.google.com/uds/Gtranslate?callback=google.language.callbacks.id101&context=22&q=%26nbsp%3Bhello%0D%0Aworld&langpair=en%7Ces&key=internal&v=1.0&nocache=1206122786074Note that the HTML entity is there sent in the request header as is the \r\n characters..

The Fiddler Response only shows the following return results. Note that the HTML entity is gone and the only the newline character is sent back (the \r\n context was lost an I think that a big deal.)

google.language.callbacks.id101('22',{"translatedText":"Hola \n Mundo"}, 200, null, 200)

I did a quick test of trying a space instead of the HTML non-breaking space and this is the result. Note the extra two spaces in front on the translation (space space Hola \n Mundo.)

google.language.callbacks.id103('22',{"translatedText":" Hola \n Mundo"}, 200, null, 200)

Rewriting the code to a more favored JS Library style

Code rewritten into a more jQuery like fashion and as I would get extremely tired of typing google.language.translate() and since the pattern/callback almost always remains the same I rewrapped it like so.The original code taken from the Google example Page

I did not let my code auto-run so I just removed that line and I removed the HTML FORM as well can the jQuery stylized everything and just placed a click event on the button. Here is the modified version of the code (in this case I make multiple calls to gt(...) to translate multiple items.

What I would consider on a live implementation

Working from our content delivery system and making some modifications to add the translator API yields this small screen snip

In this case I added only the Article Title, some Labels on the page and the main Document Body for translation. Given what I learned about the Google Translator API above I know that stripping out all the HTML, keeping the carriage returns, then translating the whole text and post translation wrapping it in an HTML PRE tag would be the way to go. Internet Explorer and Firefox seem to display the \n differently though which messes the result and some string replace of the translated results might be in order (convert the /n to /r/n) for a better unified experience in both browsers.

In my own wanderings I find that this style jQuery code is cleaner and easier on the brain. jQuery give more comprehensive bite to JavaScript code. I think I would probably use the jQuery Core Data Cache for storage of the original HTML so I can revert it all if necessary as I had that concept in my own earlier work example and before I knew about the jQuery Data Cache.

Since I did not get around to all of these things I mention I will just leave you with the end result anyway. Given the API Querystring length limitation such a bit of work would not work on a large document anyway.

There are a ton of different possible uses here from auto text chat style translations to Twitter translation on the fly or auto label text localization on data entry forms. Take your pick..

One last thought before I hit the Publish key and this goes down on the record books..

My thinking is that context aware HTML page translation or machine translation of HTML snippets is perhaps a long ways away. While I am sure there are tools you can hook that make page translation an easy thing to do and the fact that Google Page Translator services engine does a excellent job of keeping the look and feel (I would not mind seeing some notes from the Google Team on how they do that) I still find the first version of this API lacking. For the small stuff it's very cool but who wants to sweat the small stuff anyway...

Well that is about all I have to say for now and I hope these details help you in making your own decisions.

Kevin Pirkl

Parallel FX & Threading for newbies (like me.)

Kevin Pirkl (Intel) — Wed, 09 Jan 2008 22:04:28 +0000

This is my first real foray into writing threaded applications (I use worker thread's in ASP.Net applications quite often to do small tasks) and taking a peek into how much more effort it will add for managed code apps (.Net.) First let me throw out that I consider new stuff technologies kind-of a pain in the butt and perfer spoon feeding. I like what works where code is concerned and consider change as a hard truth. I hope the details and links and notes and such can help you in your own endeavors though even if this post taken as a total attempt is only a BRICK. Like I said I am not an expert but only a newbie with Threading so enjoy it or toss it. :)

Here are some quick thoughts I have, "YATTL (Yet another technology to learn)", "Clay Breshears post "This is like deja vu all over again", Rick Strahl "What's ailing ASP.Net Web Forms" Clay and Rick sum up quite well my two thoughts on new technologies, what I would gripe about myself if they hadn't already mentioned it. Many of us would like to start writing code that makes use of threading and multi-core technologies but we also want it to be as easy as using an iPod menu system (jk.) Laura Hess wrote an article titled "Slaves To Technology" which provides simple insights into why technology becomes more oppressive as opposed to unburdening. To me this means that the average developer when learning a new framework, developer tool version, etc every year or so forces them to be the beasts of burden as opposed to the other way around. Technology should make life easier and not harder or more complicated and that's all I have to say on the subject.

Learning new technologies and tools is not an easy process and typically requires a small paradigm shift for each new iteration. Modern days have given us two very big developers shifts with the first being OOP which I read about at age 13 (approximately 1978) and Parallel Programming which I would consider the second modern day paradigm shift. Parallel Programming requires some steep brain function to wrap your head around but once you get it the going get's much better.

The Parallel Programming paradigm shift is happening today and wont be ignored. Some good readings can be found by Matt Gillespie "Transitioning Software to Future Generations of Multi-Core" and Herb Sutter wrote a Dr. Dobb's Journal article "The Free Lunch Is Over - A Fundamental Turn Toward Concurrency in Software" which talks all about the switch from faster processors to multi-core architectures. Parallel Programming represents a problem for the generalist software developer in terms of language availability. Many popular languages already have the basics underpinnings for wiring into multi-core but many languages do not and may never be done in such a way as to provide visibility to the developer.

So why the headache and me complaining so much when I plan to use the technology anyway. Well lets work our way through an application and see what we find.

Here is an example of some sample code that I converted to make use of TPL and the blowback from the changes. I found an old Managed Code WinForm Migration Utility Application (just MU for short) we used to convert XML documents into HTML data for a new content delivery system we created last year. Migration Utility (MU) is a C# WinForm application that I will convert over to use a few features of the Tasks Parallel Library (TPL.) For this conversion I used the Customer Technology Preview .Net Framework 3.5 and Visual C# 2008 Express Edition and sample code found in the MSDN article "Optimize Managed Code For Multi-Core Machines" as a guide for converting the MU code. Note that it took me three install attempts on different computers to get everything working. Code segments that I'll show here parse XML files in a disk folder and downloads all the images to disk. Note that this code was not written by me (but modified) to make use of TPL and other asynchronous approaches to better see TPL in action. This was perhaps not the best example and I think something like the Quick Sort problem on the Intel Threading Challenge page would have made for a better example for comparison shoppers. In fact with TPL you might even consider the taking one of the next challenges and win $100 (see contest rules for details.) <-- Shameless plug *kp

My base computer for testing is a Dual Core MacBook Pro running Windows XP at boot.

Here is the MU application and what it did and the changes I will make and thoughts along the way.

So take some 2027 files in a folder

And process each one in a sequential fashion

First parsing the HTML contents

And then downloading the images extracted from the HTML source attributes using a synchronous download.

Now lets make changes to this application start to make use of TPL.

One item that becomes very apparent on running this web application is that the System.Net.WebClient call to DownloadFile(...) is not being done in an asynchronous manner which is easily changed. Before that change the main processing look took 10 minutes for the UI to come back and become responsive and with that change it comes back in under 10 seconds. Problem now is that with a responsive UI I need to add code to disable buttons and I have to add an indicator to show how many downloads still remain (I'll skip that for now.) One download complete callback handler to the rescue like so will suit our purposes for the time being.

So we have to wrap the call in a callback handler for the on completion event and also implement a counter to track all the calls to know when the application has completed downloading files but I still needed a polling mechanism to know when downloading was complete.

Regarding the asynchronous downloading I attempted to see if I could get better performance via setting some connections configuration options in app.config but the enhancement was negligible as the connections for images were all over the place so call blocking was not a real problem. Setting up say four connections per IP address over the normal two lent small gains but say 8/16 did not make much of a difference.

One other thing to note is the responsiveness of the UI. With the linear processing the UI just locks up and control does not come back till the entire application has completed the directory files processing. I read this little article by jfoscoding called "Keeping your UI Responsive and the Dangers of Application.DoEvents http://blogs.msdn.com/jfoscoding/archive/2005/08/06/448560.aspx" and attempted a Application.DoEvents but looking at the CPU utilization it pushed up over a constant %50 burn rate.

Don't do this.... (Perhaps except in the Winform bound timer event though.)

I tried a better approach using a thread timer callback that checked a counter and informed the user when the process was complete. Later I changed the timer to only check back every couple of seconds (initially I had set it to every 10th of a second which still pushed the CPU utilization up.)

While debugging I came across an issue/error message with multiple threads attempting to update the WinForm's list object. There is an overriding setting to ignore these errors but then your nice linear listing gets out of order. Some suggestions are to buffer the text and write it all at once when the file processing is completed. I just commented out all writes to the list object in the end but this line of code took care of some annoying non thread safe write error messages

CheckForIllegalCrossThreadCalls = false;

I decided to cut out the image processing code just to see how some loop performance numbers. Regarding image downloading taking a peeking at the task manager statistics it appears that WebClient uses Asynchronous callbacks spawned multiple to threads anyway so disabling the image downloads (now Asynch) to better check performance.

For the average run processing all the files in the folder without downloading images.

Using TPL the average run processing all the files in the folder without downloading images.

Conclusion:

Sometimes on exit the application crashes so you would need to add some checks for object existance before trying to use those objects which means more potential error handling.

Well digging the net and articles above I've read that even with multiple cores it does not mean double the speed but in some cases close to it. It perhaps kind of stinks that I did not do more but then again my only purpose was to learn a little more about TPL and I think I've managed that.

The biggest gain here is the responsiveness of the UI by just implementing WebClient using Asynchronous callbacks and TPL in this case only made things a couple seconds faster. Perhaps my bad choice of applications for this test but then again a good learning experience.

Some of my worries and lack of understanding about this new way surround thread safety. Using static variables (or not) for counters and thread timers kind of scares me as I do not understand the underpinnings in .Net on how these work and what's really safe and what's not when writing threaded code. What I did like was the ease of use of the TPL For loops and the new Lambda expressions (which you can read about at Scott Guthrie's Blog entry "New Orcas Language Feature: Lambda Expressions")

Given the changes I made the applications still crashes from time to time due to some screen objects going out of scope when exiting via the close box. All in all it was not worth the effort to make this particular application "Parallel" and it was a sort of failure but a good experience.

I like to think that the lie of "Ontogeny recapitulates phylogeny" as applied to technology might yield a better organization and core usage for Parallel Programming and a common set of base constructs found across programming languages. Even while it's a lie it's one I would still like to believe in that TBB/TPL/etc. planners could get together and organized to provide multi-core aware object constructs that act like communities making things simple once again.

Supporting notes:

Loosely taken from the thoughts of David Charbonneau's article "Hectic lifestyle catching up with Canadians health"
Clay Breshears "Eight Simple Rules for Designing Threaded Applications" provides some basic thinking guidance on good approaches when designing or writing threaded applications.

Communities like Intel's "Multi-Core Developer Community"

Microsoft's "Parallel Computing Developer Center" both provide a good start for developers seeking to gain a better understanding for both managed and un-managed environments where the developer wants to start making headway into writing code for Parallel Computing.

Other resources:

Scott Hanselman's - Hanselminutes Podcast 84 - Concurency Programming with .NET Parallel Framework Extensions

Intel's Software College - "Introduction to Parallel Programming Courses"

Intel Software Network Home -

Parallel Extensions to the .NET FX CTP -

Welcome to Intel Threading Building Blocks.org -

Microsoft's Parallel FX Blog

Google Gears - 4th hour working with Gears

Kevin Pirkl (Intel) — Thu, 04 Oct 2007 22:05:21 +0000

In memory of my friend Steve T - The wandering electron!

OK so here it is, after a bit of mulling around I finally found a good use for Google Gears! (I would find about a thousand more but they keep me locked up in my cube with a stack of ToDo items.)

Use Case for a web application distance problem: There are times that a person wants to write a web application that works in distant geographies like from China to the USA and still maintain excellent performance. Consider a Content Management System (CMS) administration tool that spans multiple geographies.

Here are a couple approaches (I sure there are a million more)

One approach would be to place the CMS administration tool in one place have all geographies work there. Nightly copies of data would be delivered to a Content Delivery Network (CDN) hosting a web front end like the Akamai network. A slight variation on that theme might be to generate static files derived from the CMS tool delivered to the CDN for even faster performance.
:-)
Another more distributed approach might be to place server racks in multiple geographies across the world and then create nightly data synchronization routines to bring data back to a central server for final distribution back to servers (insane from an administrative standpoint.) Each geography would own and maintain it's own information and a copy of all other geographies data. Administration tool would run fast for sure and the delivery vehicle in each geography would be faster. The trade off here is that system application maintenance would be pain in the a$$. Imagine having to make a single change and distributing that change to 14 locations not to mention the cost of updating the data synchronization routines.

Google Gears to the rescue - Application performance gripes in this case scenario can be boiled down to the parts of such a system. What are some internal pieces we might see in such an application approach with ISN.

Localization support patterns and distinction. Each page needs to be able to be translated (potentially) into multiple languages. Virtualized URL pattern support for localization and display.
Rich Text Formatting editor (RTF) that can enforce style and formatting usage. Pages should share a common look and feel and shared functionality. Ability to embed JavaScript and HTML forms.
Categories and Page types for classifications. More ways to sort, compile and report the data as it is consumed by clients.
Custom navigation options editor. Links and menu options that apply to specific content or even groups of content.
Lists of non-translated articles for each geograph or a work list. Search and report on articles by category, type, language code etc.
RSS by category and language support.

There are more but I am most concerned with the slowest parts of this style of application and why single point delivery is a problem. Out of all the above aside from distance based latency large and complex pages and the biggest problem. Take for instance a nice RTF editor like FCKEditor. Put this tool on a single web page and deliver it from the USA to China and see how long it takes. There are about 10-15 subfiles that comprise the FCKEditor experience (just a very rough estimate.) FCKEditor files are pretty static and using Gears Local Server Module API give an excellent resource cache for all of these files. Using the Gears LocalServer class container to create a ManagedResourceStore that contains these 10-15 controlled by a simple manifest file means mighty groovy goodness! If fact the time it will take me to write up this blog post will be much longer than it took me to code the solution to the problem.

I ran into a couple hangups here with some version problems with Gears sample code and the main download page. I had this same problem the first time around on my own development box but have sorted it out. If you happened to install Gears from the main link http://gears.google.com/ then you should just uninstall it and go to this "Gears .2 release" link instead. From my experience it appears that they might have updated the sample code to work with the new release and it is not compatible with the older version. I installed from the first link on clean server boxes for in Russia and China for testing the performance boost and both had the same issue which was solved by re-installing from the .2 release. Update (Oct 15 2007) Addition - I get something here at least why my code seems to have some issues. My problem here with Gears versions is that I’ve been working from the Beta site install version code or .2 versus the main site downloader. http://code.google.com/apis/gears/install.html versus http://code.google.com/p/google-gears/downloads/list. I’m just not sure all about it but it appears that the sample code has been updated to use the second link or version .2. KP

One other problem that I can across is that I wanted to add a little bit of code to call openStore(string name, [string requiredCookie]) to check to see if the store existed so I could enable/disable buttons on the screen for a clean user experience but the calls to openStore always returned null for me even though I can see that it is working correctly. Not sure what I might have done wrong on that end but I will take a better look at the code tomorrow.

The RTF editor was the biggest hindrance to the geographic distribution of this application and Gears solves that problem. It's worth noting that some other changes like HTML reduction via style sheets and dropping viewstate from the ASP.Net pages has made a big difference in overall search performance as well but those are often issues dealt with in a second go around with the code. What I like about all this is that the page that took 15 seconds to load in China now takes about three.

So in three hours of play and one hour of coding I have all the code up and working in production and a viable future implementation change that will end up saving thousands of dollars in application maintenance costs (and developer headaches later.)

I just want to say Google Gears Rocks! For more resources check and watch the Gears API Blog

Kevin Pirkl

Hacking Intel - XSS Security exploit with ASP.Net using .RewritePath and Request.RawUrl bypassing ASP.Net native script protection

Kevin Pirkl (Intel) — Thu, 20 Sep 2007 22:46:16 +0000

Hacking Intel: XSS Security exploit with ASP.Net using .RewritePath and Request.RawUrl bypassing ASP.Net native script protection (.Net 1.1 and 2.0)

What is XSS: http://en.wikipedia.org/wiki/Cross-site_scripting XSS also called Cross-Site Scripting is a web based security vulnerability by which a person injects HTML or Client side script code into a web page without the knowledge of the originator of the web page.

Web 2.0 enhanced GUI experiences and RTF editors go hand in hand and the increased support and ability to POST HTML content (and to protect against malign intent) means that XSS becomes a more serious issue on a daily basis. Practically weekly a new hack or workaround enabling a user to inject malign content gets found.

Any web page that can take as input QueryString or POST input like and echo's it back can practically be considered a XSS exploitable site. Other simple variations on this theme are described but another simple one that I will consider here is

Securing ASP.Net: Typically ASP.Net web page developers rely on one or two mechanisms to prevent XSS issues. The fist layer of protection is found already set on in most machine.config files.

This is a first level of protection and can be explicitly set in web.config files or down to the @Page level for making allowances of HTML RTF editor posted content. With validateRequest set to "false" we now start to enter the realm of the XSS exploitable web site. From this point on it is would be considered on part of the developer to scan and remove CSS includes, JavaScript:, and using the .RewritePath(ShowBlogPostings.aspx?year=$1&month=$2) I echo the Request.RawUrl to the web page and BOOM!

Doing some more isolation work on this XSS exploit to determine why it was getting by I've found that if my re-write path is plain or without QueryString parameters to the page i.e. "ShowBlogPostings.aspx" that ASP.Net responds with a proper error message (A potentially dangerous Request.QueryString value was detected from the client (a="