Job Security Opportunities

By Clay Breshears (Intel) (86 posts) on October 13, 2006 at 3:19 pm

It's never a good time to lose your job. Unless it turns out to be the push you need to go off and do something more worthwhile, more personally rewarding, or offers more money. At Intel, there have been reductions in the workforce over the last few months. In times like this, it's nice to have either job security or the knowledge that there may be a place to land if you're one of those chosen to test the employment market. Since it is very rare to have the former and the latter can at least make the suspense bearable, let me point out a couple of items I've seen recently that might help you sleep a little easier.

Results from a survey by the Simon Management Group of 500 HPC users are reported in Electronic News. It seems that the lack of parallel programming experience is keeping scientists from making the most use of parallel supercomputers. This is slowing the progress of scientific innovation and discovery. Apparently, folks have been using very high level languages (Mathematica, MATLAB, Python) to implement their computations. Running small problems presents no problem on desktop systems, but these platforms can't easily handle the tens of gigabytes of data for typical real-world problems (let alone the hundreds or thousands of gigabytes for tomorrow's data sets). Thus the need for HPC.

First off: Duh! Parallel programming is difficult. MPI is one-step above an assembly language for distributed-memory computing. Secondly, while I agree that user interfaces to HPC haven't really made much progress toward usefulness by the "common" scientist, I have seen some domain-specific attempts to make things easier. However, the bottom line is that someone has to program these machines (even if a very high level language or web interface were the way to interact with the hardware). I've not seen a university course offering in Fortran Programming in over a decade. There is still a lot of legacy Fortran code out there. I can see why researchers today find programming parallel machines difficult.

Therein lies the opportunities for Thread Monkeys. One, thread those Very High Level Languages (Parallel Python => Hydra) and desktop computation tools to run on multi-core processors. Hard drives have gotten huge, so I don't think the actual size of the data sets is the stumbling block, but rather it is the processing horsepower needed to handle such large problems. Two, help these scientists take their desktop models directly to the Big Iron machines. This may only be a stopgap measure, but there will always be someone that needs very fine control of their computation to get best performance. Three, develop the interfaces that scientists are accustomed to using on their desktop systems but that run on HPC platforms. This is the "Star Trek" solution. Being able to verbally state your problem and have the computer do all the data collection, collation, and computation will be the ultimate goal for this activity. If you can live long and prosper, you could have job well into the 24th Century.

On another front we find out that COBOL is not dead. (That's the COmmon Business Oriented Language, not the D&D monster.) Ten years ago the furor of Y2K was really starting to make an impact on users of systems that relied on a fixed-size field for the year. Payroll, accounting systems, and similar applications had been written with a two-character field to contain the year. Programmers in the '70's never expected their codes to be used into the next millennium. They didn't take into account upper management's positions of "if it ain't broke, don't fix it" and "why pay someone to re-write something that still works." For the Y2K conversions, there were two hurdles. First, the current data records had to be expanded to hold four-character years. Second, the programs that would be using this new data needed to be made aware of the change in the year fields so that they wouldn't be reading incorrect values. COBOL programmers were making some pretty pennies in the run up to the year 2000 modifying the fixed field record descriptors in programs.

A new survey by Computerworld has found that 62 percent of the IT managers surveyed still use COBOL in their data centers. While 36 percent have stated that they are moving away from the language, it has already been recognized that the process will be painful and expensive. Remember those managers from the previous paragraph? Well, their successors have the same attitudes. The painful part is that more modern alternatives (Java or C#) are not as well suited to the types of computation that COBOL is good at. Estimates of the increase in code size have been reported as high as five times the current number of source lines. The expensive part is due to the fact that the programmers from the '70's, that did nothing but COBOL, have retired. So, new programmers are going to take a relatively long time in order to understand the current codes and be able to convert them to a modern language. That's if you can first find a programmer that understands the finer points and intricacies of COBOL. (Surely, not all of those Y2K COBOL programmers that made billions have retired to the seclusion of their private islands in the South Pacific.)

What's the opportunity here for a Thread Monkey? I guess I don't actually see one, unless someone were to create parallel COBOL (right after object-oriented Sanskrit on Rails). I just found this story fascinating since COBOL was the first programming language that I learned. If worse came to worst, I suppose I could dust off my COBOL skills and get into translating COBOL to C# or Perl or Ruby. The rest of you will just have to work on that Star Trek idea.

--clay

Categories: Multicore

Comments(6)

November 3, 2006 10:10 PM PST


mike		Clay,

Next time you read this blog, can you get a hold of me directly at michaelmallen@yahoo.com.
We are officially changing over to corporate e-mail, but yahoo has worked well for me, I like it's simplicity.

I want to get your address & send you a Language that will end all headaches, when it comes to programming in parallel, we call it TPP, True Parallel Processing (Parallelization).

We are going to start making it available to the masses anyway, but your so into it *thread monkey* I just believe you will really appreciate the language. We have the kernal, tools, file system etc., but the language will give you plenty to digest.

And I believe it's current size is 4KB, a weekend's read.

And guess what, our present design is set for the X86 platform. We plan to get around to IBM Cell Core & others real soon though.

Anyway, I think you will get a real kick out of the work our guys did.

I happen to be going to a party in SF tonight (don't call it San Fran or Frisco). The 'head honch' from INTEL is the co-chair. INTEL has a lot of SF connections, all the ex executives resided/or were raised in the city by the bay.

Hope to hear from you soon.

Take Care,

Mike
Virtual Parallel Systems
CEO
November 3, 2006 10:15 PM PST


mike		By the way Clay, wasn't COBOL written by a woman? I was talking about this just last week with my top guy.

Mike
November 24, 2006 6:08 PM PST


symbolset		Grace Hopper is held to be the "mother of COBOL" because much of it is inspired by or derived from her language, FLOW-MATIC. You can learn more at wikipedia: http://en.wikipedia.org/wiki/COBOL

From "The Tao Of Programming" : http://www.canonical.org/~kragen/tao-of-programming.html

Each language has its purpose, however humble. Each language expresses the Yin and Yang of software. Each language has its place within the Tao.

But do not program in COBOL if you can avoid it.


Grace Hopper was notable for a great many other things as well.

To speak of a programming language as "good at" a particular kind of calculation is a little silly, unless you're talking about a language like APL, which has vector array data types and operators for them. APL related languages would be grand for exploiting the benefits of threads.

The article Clay references discusses the use of COBOL on NYSE mainframes. An APL derived programming language K is also used for this, in conjunction with a proprietary database called kdb+. Another derivative of APL called J is free and available at http://www.jsoftware.com/. That website lists Intel as a corporate user of the language, so you may have it available locally.

On topic: Having just changed jobs myself I can agree that change is often good. So often in life we just muddle through. If you keep your contacts current and your eyes open for the appealing opportunity you can often move closer to the work that meets your career goals or that you would find more fulfilling. We should all take the time to do that.
November 27, 2006 3:10 PM PST

Clay Breshears (Intel)
Total Points: 7075
Status Points: 7075
Black Belt
Ever since I heard about her, and especially after I'd met her, I've always admired Grace Hopper and her career.

I haven't thought about APL in many years. I remember looking through an APL book at my local B. Dalton's while I was still in high school. It was fascinating to look at the symbols used in the language but it was too confusing for my young mind, then. Still, I'm not sure I'd want to tackle learning it today.

I understand the power of functional languages and expect there would be many opportunities for parallelism and threads. I don't know if an explicit API would be the best solution, but something behind-the-scenes that would harness thread pools could realize performance advantages. I'd think Prolog, too, might even be able to implement parallelism in the interpreter to use threads.

Good advice on changing job and advancing toward a career. On the COBOL front, there is an article in the Financial Times (online access needs a subscription) that says, even today, new applications are being developed in COBOL and other legacy languages. A large percentage of experienced mainframe programmers are going to be ready for retirement next year. Since universities no longer offer programming courses in legacy languages, companies are doing internal training or trying to convince local colleges to offer courses.

If you can't "do," maybe this is the chance to "teach." I'm not too surprised that someone like DeVry or ITT Tech don't have courses in COBOL programming, just to serve this market. Fortran is going to be another language that is not being taught, but there are billions of lines of legacy code, especially in technical or scientific computing, still in use and needing to be maintained and updated. Standards committees are still cranking out updates to the Fortran language spec, too. At the recent SC conference, there were quite a few Fortran vendors in evidence. (See the Doctor Fortran blog.) Legacy languages aren't dead, they're just another opportunity.

--clay
February 23, 2007 3:23 PM PST


Intel® Software Network Blogs » Blog Archive » Do We Need A(nother) Parallel Programming Language?		[...] Unfortunately, the 800-pound gorilla sitting on one end of the teeter-totter is the billions of lines of C, C++, Java, and Fortran code.  Who is going to be willing to translate their 2-million line application into a new language and then write in all the new coding features to implement parallelism?  (Personal note to current Thread Monkeys willing to be the future Parallel Giraffes: Here’s another activity that can translate into years of job security.) [...]
February 23, 2007 11:22 PM PST


Intel® Software Network Blogs 		application into a new language and then write in all the new coding features to implement parallelism?  (Personal note to current Thread Monkeys willing to be the future Parallel Giraffes: Here’s another activity that can translate into years ofjob security.) So, for now, I’m content to teach people about how to thread, how to avoid the problems common with multithreaded code, and how to write code to scale the number of threads and the application execution with the increasing number of cores.  It


Leave a comment

Name (required)

Email (required; will not be displayed on this page)

Your URL (optional)


Comment*