Batch processing’s value to Web developers

 

One of the big shifts since the beginning of the Internet era is that applications have moved to real time data processing. Many developers have never written anything with the batch processing paradigm, thanks to changes in education and what is experienced in the business world. All the same, batch processing is still a valuable technique that has a place in a developer’s toolkit.

What is batch processing?
Batch processing is when your code is designed to iterate over input that has accumulated since the last time it was run; it is also designed to be run on a regular basis. For example, a bank will record transaction data throughout the data, and then use batch processing at night to post the transactions to customers’ accounts. This is in contrast to real-time processing, where the calculations are a direct response to the input occurring.

I first learned about batch processing in high school, when I was taught to write in COBOL. To this day, COBOL applications are still heavily used in many batch processing scenarios, including end-of-day processing for banks, insurance companies, and other similar big-data environments. But there is no reason for batch processing to be limited to mainframe-like applications; for example, a service that has a recurring billing component or perhaps a scheduled reporting system needs a batch processing system.

Advantages to using batch processing
Batch processing has a number of advantages over real-time processing, including the following:

  • Transactions can be much faster since they do not have to be processed up front; instead, just enough data needs to be provided so the batch processing task can find the data it needs.
  • If there is a period of the day when your company is less busy, your batches can be run during the slow time, leveraging otherwise idle infrastructure, and requiring less of it to handle peak load times.
  • Batch processing lends itself very nicely to virtualized and cloud environments where resources can be scaled or transitioned to loads as needed; this can save serious amounts of money on hardware and power.
  • Batch processing is great in situations where the application has unusual resource needs because the systems that interact with the users can be separated from the systems that do the processing; again, cloud and virtualized environments come to mind.

Why has it fallen out of favor?
If batch processing is so great, why is it slowly disappearing? Web development has a lot to do with it, but the changing face of business in general has a role to play as well. Here are the primary reasons:

  • Web applications cannot start themselves, they need a trigger to start, making the idea of scheduling a batch non-intuitive.
  • Web applications are expected to be deployable via file copy only, which nixes the idea of an OS-level scheduler.
  • Web applications are supposed to perform small, discrete amounts of work and return results in a few seconds; one cannot easily use a user’s action to trigger any kind of batch processing.
  • Companies are doing business on a global basis, with no “overnight” anymore or little downtime.
  • Users expect instantaneous results or transactions to be reflected immediately.

Various approaches
While I haven’t programmed in COBOL in quite some time, I have written applications that used batch processing on a regular basis. When working on a Web application, I have tried two approaches.

The first path is to have a portion of the Web application perform the batch processing and to have a scheduled job retrieve that page as it required. While this works, and it allows you to easily leverage the rest of your code base, I feel like there are too many moving parts in the scenario, especially when it can be a long-running task.

The other method I’ve tried is to write an entirely separate application that can be called from the system’s scheduler. I like this approach much better, but it only works when the batch processing task can have access to the main application’s logic (usually through a shared library) or does not need that access.

There are other choices too; for instance, databases have the ability to schedule jobs, and if your processing can be fully contained within the database, that may be an excellent option for you. Some development frameworks and systems (like the OutSystems Agile Platform, which I have been using a lot lately) have the notion of batch tasks built-in, which makes life easy. It’s also possible to run your system continuously (as a daemon or a service) and use an application-level timer to trigger the processing as-needed.

However you go about it, though, you will need to make sure that your method can be scheduled the way you need it to be scheduled and that it works well with your deployment choices.

Blue 2009 Bollywood Movie

A treasure lays buried deep at the bottom of an ocean. There are many people who want to get there by risking life, be it theirs or someone else’s.

‘Blue’ is touted as Bollywood’s first underwater, action adventure thriller. Directed by Anthony D’Souza , the movie is inspired from Hollywood flicks like ‘The Deep’ or ‘Jaws’, but it is not a copy.

The film is about a lost treasure, which three friends try to find undersea. Akshay Kumar , Sanjay Dutt and Zayed Khan play the respective roles.

But in this hunt for the treasure, there’s deceit and betrayal by someone in the group.

Lara Dutta is paired with Sanjay Dutt in the film. The actress did not know deep sea diving before signing the film and underwent a rigorous Scuba diving training to prepare for her role.

The movie’s second heroine Katrina Kaif doesn’t have a big role. But director D’Souza says Kat will surprise the audiences with her role. Sunil Shetty also has a brief role in the film.

Most of ‘Blue’ has been shot

in the Bahamas. Akshay, Sanjay and Zayed swam with real sharks while shooting their stunts in the film.

While shooting one such stunt under water, Akshay Kumar had a narrow escape when he accidentally hit his head against the metal of a sunken ship and began bleeding. Blood attracted the sharks, but before any mishap could happen, the actor was safely pulled above water.

To tone up his muscles, Sanjay Dutt hired the services of a special body building trainer from the US – Lawrence Gonsalves, who guided the actor on losing his former beefy body and developing a leaner physique.

Special water-resistant make-up was required for the film’s leading lady Lara Dutta. For that the producers roped in Hollywood make up artist Nikoletta Skarlatos.

‘Blue’ also feature a special appearance by Australian pop sensation Kylie Minogue who has not only sung for the film but also shot a sizzling video with Akshay Kumar.

Oscar winner A R Rahman has composed the tunes of the film’s songs. In fact, Kylie Minogue agreed to sing and dance to her song in the film because she was captivated by Rahman’s composition.

The movie apparently also has an underwater song picturized on the entire cast.

With a budget of over Rs 100 crores, ‘Blue’ is touted as Bollywood’s most expensive film to date. Made under the banner of Shri Ashtavinayak Cinevision, the movie is set to release in May, 2009.

Chrome Frame: Google Offering plug-in to convert IE into Chrome

Google has developed a plug-in for Internet Explorer (IE) that will allow Chrome to run in IE. Chrome Frame is a new browser plug-in developed by Google which will give user a chance to feel Chrome browsing experience inside Microsoft’s Internet Explorer.

Many experts have claimed that IE has fairly low web standards and is susceptible to virus attacks and lacks in performance as well. Many users are turning to Mozilla Firefox, Google Chrome or even Apple Safari for safe browsing. But that does not change the fact that IE is still the largest used web browser (as it comes integrated with Microsoft operating system).

Many corporate houses still run IE. But after the launch of Chrome Frame things will change dramatically. Chrome Frame creates a new frame inside IE and with this frame Google packs all the essential data in a lightweight plug-in to convert IE into Chrome. Chrome Frame also injects the latest versions of Chrome’s Webkit and JavaScript engines into IE. Google claims that its goals are noble and they just want to provide users with better performance browser.

While it is true that running two browsers in one will take more resource from computer but Google says that Chrome is designed to give resources back to the machine when users are not using them with the browser.

Google claims that it won’t be explicitly advertise Google frame in IE but will use subtle methods to alert users of its existence. For example, if users browse to a Google app in Internet Explorer that may render better in Chrome, Google might have a message on the page informing about the plug-in.

Chrome Frame is still in testing stage and will work with IE6, IE7, and IE8 on any Windows-based machine. Microsoft has still not made any comment on this but there are chances that Microsoft will ban Google Frame.

Conficker: Experts from all over the world fail to defeat internet worm

Till now no one can even think that a worm could knock down global internet. Even tech experts have failed to conquer this deadly worm called Conficker, which has already infected five million computers, and may soon crash the internet in all countries. So dangerous is the threat, that the world’s largest computer security companies have joined together with governments around the world in an unusual alliance to pool their resources and solve the problem. So much so, that a group (Conficker Working Group) to defeat the worm has also been created. The worm, which was first detected in November, last year, spreads rapidly to computers through a flaw in the Windows operating system. Infected machines are co-opted into a “botnet” army, which can be controlled and used by the hackers to launch unprecedented cyber attacks. “The general agreement in the security world is that Conficker is the largest threat that global internet is facing from a cyber crime point of view. It has proven to be extremely resilient. It’s almost impossible to remove,” Rodney Joffe, a Director of the Conficker Working Group, told Sydney Morning Herald. “The best minds in the world have not managed to crack the code behind this yet.” Microsoft has offered a 250,000 dollars reward for information leading to the identification of the individuals – or rogue governments – behind Conficker. But so far international effort to find a solution has yielded few results. The creators of the worm can do anything they want with the infected machines including stealing users’ banking details or flooding government servers to knock them offline. The worm has a built-in mechanism to prevent people from scanning their computers with anti-virus software, says Joffe. Unfortunately, even for those who wipe their computers clean and start fresh, if they back up any important data on a portable hard drive, the clean machine is re-infected when the drive is connected to the computer. The worm also spreads automatically between computers on a network and infects machines without the user having to do anything other than switch their computers on. While majority of the botnets can be destroyed by disabling the server used to issue commands to infected machines, but with Conficker the location of this sever changes every day and state-of-the-art cryptography means it’s almost impossible to crack.

Modding MOTOROKR E6/E6e/A1200

For the growing interest of how to mod your MOTOROKR E6/E6e/A1200 i have published a article How i moded My MOTORKR E6 edge aka E6e. I have also published Short tutorial for flashing your E6/E6e.

See this and let me know about your progress and never hesitate to post if you face any kindof problem.

Happy modding!!

Finally I modded My MOTORKR E6 edge

Hi Guys!! For since somedays i was busy with my newly bought MOTOROKR E6e aka E6 EDGE. I was too curious to mod it by as you know for modding and installing apps you have to flash your firmware to some firmwares which supprt modding. Like I have 22P firmware installed by default and these firmware is only available in indian phones, as far as i know. So i was eagerly waiting for the 22P firmware to come on internet as i have the alternate way if it doesnt work. After i got the original R535H10_G_11.87.22P aka 22P firmware. Now what should i do. To know more go here.