Sunday, July 08, 2007

Database scalability with MSSQL 2005

Some news from the battlefront...

We are expecting at least a 75% increase of our internet traffic, db transactions, network and database activity in general so our entire company is working hard on getting ready to manage this incoming wave.

Regrettably, life is not always good and the core of the business is setup around a system using a typical client/server architecture where each workstation creates a connection to the database and to make it worst, establishes server side cursors to fetch all data they need.

Now on the other hand, the website is driven by old school ASP pages, using adHoc Queries or direct stored procedure calls to the already beat up database. Finally, it is important to mention that this is running on ODBC connections.

End result?, chaos, in high traffic moments the entire network slows down and all the applications switch to a slow silent death walk to a total IT crisis. There is no caching techniques, no business layer, basically no middle tier... all the opposite things to what I'm used to deal with, and that is why it is my mission to turn this around and take over the world. (moooooahahaha)

Well, the mission is simple, we can't change much in short time, so we need to make this system work as good as possible. So, my first task is to provide a list of recommendations towards the deployment of our new database servers running MSSQL 2005.

Yup, we are migrating from MSSQL 2000 to MSSQL 2005, people will think that it is a simply straight import/export, but special considerations need to be made when dealing with the databases collation which has completly change from one version to the other one and to the famous "compatibility modes". Keeping an imported database in Compatibility mode 80 (MSSQL 2000) may save you headaches but will offer you little in terms of all the good new toys and performance tricks that you can apply on compatibility mode 90 (MSSQL 2005).

Bellow is a summary of my initial recommendations aimed to the physical database storage design aspect that you should follow when dealing with a situation like the described above:

  1. Server side cursors implies lots of memory usage on the database. So, increase memory on the database server, the more the better. Make sure your database is using it, by checking the database memory limit. (special attention to the 3gb limit on the non Enterprise versions of MSSQL)
  2. Multiple core machine? well, you will not use them if you don't separate that big nasty .mdf file into multiple .ndfs, what is the recipe? the number of data files within a single file group should equal the number of CPU cores.
  3. For optimized I/O parallelism, use 64 Kb or 25 Kb strip size when defining the RAID configuration.
  4. Use manual file growth database options. Automatic is only for development (ja! you didn't knowt that did ya?)
  5. Increased size of “tempdb” and monitor space usage, adjusting accordingly. The recommended level of available space is 20%. All your temp tables and indexes are created there so, keep that guy with enough space, if you are using the default size that is only 8MB, and that is basically 8 floppies, so be nice, put some more space there.
Alright, so that should buy us some time to concentrate on the next stage, database optimizations. Until the next post.

Wednesday, June 20, 2007

CodeGear drops C# Builder.

I was listening to one of the weekly community podcasts from CodeGear, and one of the topics that was mentioned was the announcement of C# Builder being dropped from the CodeGear Developer Studio 2007, to be released later this year.

Winforms seems to be a no no too. It seems they will be putting all their efforts on Delphi and C++ and Delphi.Net using the VCL.Net. They will convert their existing support for C# Builder to the same level of the VB personality, which means edit, syntax highlight, compile, and basic
debug functionality.

It looks like we will have the possibility to use any existing C# component and add it to your Delphi.Net project with no problem.

For people using ECO with Winforms, they will be ways to move into the direction of VCL.Net, which will be fully supported by ECO on the coming Delphi.Net release.

I certainly agree with their moves, there is no point on supporting a language that they cant controlled. They can use the .Net framework or the Win32/Win64 native APIs, but just like they did with Win32 by building the impressive VCL on top of it, expanding the VCL to support .net will also forced the VCL under native APIs to be extended to new cool functionalities.

Lots of news around CodeGear, which just reasures my thoughts towards the fact that these guys are doing the best they can to have things going the proper way.

Oh final comment, initially it was mentioned that Generics will be only supported from a consumption level, that seems not to be the case, production of Generics will be there too. But, only for .Net initially.

Thursday, June 14, 2007

ASP.Net not too scalable....

Scalability is a daily concern on my business. We managed thousands of requests, that startup lots of transactions that will make any online bank website simply jealous.

A couple of times, I worked with companies that use ASP.Net as their web framework of preference and it was not long when we start having scalability problems. It is a well known issue with ASP.Net, but there are solutions.

I found this interesting article that talks about the issue and gives a good solution.

Read it!

Wednesday, June 13, 2007

Using Visual Studio 05...

Yes, I know, but hey, I was programming on Java a couple of days ago, so for me it is just more stuff to learn and at least it is more visual than Java.

I'm in no way leaving behind my sexy Delphi, we just got a new update and a cool roadmap, but, I need to develop some web applications that connect to my Delphi middletier and the option I have right now is to do some ASPX on C#.

If I'm lucky enough, I should have the new Code Gear Developer Studio with support for .Net 2.0/3.0 by September, so I can use it.

Monday, June 04, 2007

The big reunion.

Hi guys,
I don't know if you were aware of this great reunion between the last two old schoolers of the software industry: Bill Gates and Steve Jobs, on an interesting chat together at D5!!.

The vision they had and have is simply outstanding, the advances we enjoy today on the software area can be easily blame to these two great minds.

Enjoy!

Friday, June 01, 2007

What will many cores mean to future window releases?

This is a good article about what is going on with Microsoft around the new trend on Multi Core devlopment.
You got to love AMD Opterons :)

Thursday, May 10, 2007

Faster web pages...

This is the link of an outstanding presentation made by the Yahoo’s “Exceptional Performance Team” about optimizing website performance by addressing front end issues. This is part of the web 2.0 expo that happened last April. A MUST read.


Enjoy,


http://www.web2expo.com/presentations/webex2007/souders_steve.ppt

Yes, it is a power point presentation.

Sunday, May 06, 2007

A great option for middletier development!

For all those developers that are looking for another option to the mighty DataSnap, now there is a great opportunity to develop reliable and scalable middletier solutions for the same price.

Components4Developers is offering their popular middletier software for free. What is the deal?

Simple, register and download. TurboMiddletier is a full version of their enterprise middletier software KbmMW (a couple of transport protocols and channels are removed but nothing major), it only supports the latest Delphi version, so, if you have Delphi 2007 go and get it now!!

I personally use Data Snap, RemObjects and KbmMW in our projects and can only say good things about them. I do admit that KbmMW is a more mature framework with lots more functionality, but on RemObjects SDK favor, they offer a great DataSnap integration pack that help companies using DataSnap to migrate to their SDK.

Whatever you pick, you will be on good hands, in the meanwhile go and get that!

Saturday, April 21, 2007

Using Agile methodologies on your favorite IDE.

My current job is moving towards an Agile development, so, in order to be one step ahead of the game, I am implementing on my home projects most of the methodologies that we will or should be using in our everyday development life.

Browsing around I found this site, which basically is dedicated to implementing Agile methodologies in Delphi. Take a look at it, and subscribe to the forum, it promises to be a great source of knowledge for all those XP/SCRUM/Agile fans.

Sunday, April 15, 2007

Ajax and Delphi

I'm finally working on the web applications of my home project, and I decided to work with Intraweb 9 as my web framework which now supports "Ajax" events on most of its components and comes built-in my brand new shinny Delphi 2007.

But, browsing around I found a set of extremely interesting examples and articles about using the Microsoft flavor of Ajax (Atlas 2.0) on ASP.Net with Delphi.

Take your time on rewiewing these examples from Steve Trefethen.


It is worth to mention that TmsSoftware replied to my question about possible updates to their Intraweb web components, the answer YES!, so lets wait a couple of days, for the goodies to come.

Saturday, March 24, 2007

Delphi Developer job offer in NJ?

I saw this add from "The Lair of the Pythia" blog, so I will help spreading the word.

Delphi Job opening here!

So go there and make us all proud!

A painless self-hosted Git service

Remember how a part of my NAS setup was to host my own Git server? Well that forced me to review options and I stumble into Gitea .  A extr...