Unstructured Data can create chaos
June 12th, 2013
It seems that no matter where you go these days, twitter, your favorite tech blog, email newsletters, that everyone is talking about big data. Let's face it, big data is a buzzword, or buzz term, that many technology professionals are being forced to address. Most database guys like myself have been dealing with large amounts of data for years. What was once kilobytes of data turned into megabytes, then gigabytes, then terabytes and now even beyond that. We've dealt with this sizable data in a lot of different ways including table partitioning, regular archival and purging and the creation of data warehouses that are away from our regular transactional databases. We've had the time to analyze what is coming into our databases so we can transform it into something useful. This latest wave of "big data" is taking some of these approaches away from us for a couple of different reasons, velocity and volume.
At some point the size of the data becomes just too big to handle and the speed at which it is coming at us is too quick for our systems to handle. Now, fast forward to the wonderful world of unstructured data. This world states that we really don't care what the data that comes in looks like we'll just store it. Then after awhile we'll be able to do something useful with it. But just how realistic is this approach? As a database professional, I like to ensure data quality. By introducing unstructured data into my world you've thrown a lot of my ability to ensure data quality out the window. I can store it for you. I might even be able to query a lot of it and produce useful insight from it but over time the data just becomes more and more difficult to manage.
For example, once I've traversed the last 2 years of web logs and created a dashboard of how often our customers go to each of our web pages do I keep the detail information just in case I might come up with a new way to traverse and create new business knowledge? If I do keep it, do I tie back my new business knowledge to the rows of unstructured data for purposes of drill down? In some shops this may be impossible. My only real option may be to archive it because while I'm analyzing the bulk unstructured data that is stored let's not forget that all my current customers are quickly producing mounds of new data that I'll have to do something with sooner or later.
To be fair, vendors are giving us ways to deal with this data. Newer, open source, database technologies such as noSQL and CouchDB (a derivative of NoSQL) are document based solutions. The Hadoop File system (HFS) provides file based storage that is, in theory, easy to get to and designed to store bulk data. Developers are slapping SQL like interfaces like Hive on top of HFS in order to facilitate those of us with SQL skills access to the data in these new systems. But wait, if it is in fact truly unstructured, how do I know what I need? If data is coming in from multiple sources and just dumping away into an open file system how do I make sense of it?
Well, this is where the database guys come back into the picture. This is also why, in my opinion, the relational database management system is not going anywhere soon. Extraction, transformation and loading (ETL) techniques from these large unstructured data sources will still need to be written in order to make the data into usable and business ready forms. The data will need to be tied to valid business entities such as users, clients or customers or real assets such as servers and/or data centers. Without the knowledge of what a piece of unstructured data is directly tied to it will be difficult, if not impossible to derive any real value from it.
Let's also not forget the big RDBMS guys like Oracle and Microsoft are adding big data features into their tool sets. Either that or they are buying companies that already have tools and creating hooks back into their flagship products. SQL Server has introduced PolyBase with its 2012 iteration that promises to tie unstructured big data with its relational counterparts. Oracle has its own NoSQL database and a fully configured Big Data appliance that is ready to capture your organization's data. These are all relevant and good approaches to the problem but without data order there is data chaos.
You, and your team, must take a systematic approach to what data you are capturing and why. Then you must consider its value to the business. Each data have a relationship to a business unit or units within your organization. Once the data are categorized, governance rules must be created. You can't simply say we'll keep all data for all time, that is not realistic and will eventually create chaos. Data retention rules apply here, they may be self imposed or perhaps government imposed in the case of financial institutions but clear rules should be defined.
Unstructured data cannot always be handled in real time either. Clear rules should be defined around which data needs to be handled in which order. If the data is directly relevant to revenue then obviously it is more important and should be handled with the fastest available applications and hardware. This is where defined disk tiers can come in handy. If data is not needed to be instantly accessible it can reside on older, slower, commodity disks. But if data is needed in real time or near real time then perhaps solid state devices are needed.
If you haven't already been approached to deal with unstructured data, you soon will be. I hope that this has helped you think about some of the pitfalls of dealing with it. Do yourself a favor and start familiarizing yourself with some of the free tools that are out there. Cloudera, as an example, has a free Virtual machine you can download that has a full install of Hadoop with Pig and Hive on it for practice sessions. With a reasonably powered laptop you can get this up and running in a half hour or so. If you don't, just keep up with industry blogs and whitepapers to stay in touch with what is out there.
Here are links to some of the stuff I talked about in this blog
CouchDB
Hadoop
SQL Server 2012 and Polybase
Oracle Big Data Appliance
Oracle NoSQL Database
Cloudera Quickstart VM - Cloudera Quickstart VM
Bill Schoonmaker
Data Architect, EMC Corp
Wednesday, June 12, 2013
Friday, January 4, 2013
Welcome to the FitBit Generation
I'm overweight. There, I said it. According to some charts I'm obese although I don't think about myself in that way. I've gone up quite a few pant sizes since my college days. Then I was a svelte 170 pounds and was pretty active in High School. I never was much of a runner, just didn't care for it. I did like to lift weights but never really bulked up in any way. I guess I was just "normal" for my height of about 6'1".
The years tend to take a toll on us if we don't continually maintain ourselves. For me, they've been particularly heavy on the pounds. So here I am at the ripe old age of 43 and staring an overage of about 75 pounds dead in the face. There are some serious health problems that can and will begin to crop up when you're pushing around that much extra weight. To be fair, like I said, I don't "feel" like the fat guy in the room most of the time but I do notice that in most cases I am probably the heaviest guy in meetings or in general gatherings with my friends. Even saying that in writing makes me know that something has to change.
I've dieted but only one time really seriously. About 2 years ago before I started my new job I was a member at the Gym where I worked and joined a bootcamp program. Three times a week I got up at 6:00 am to go exercise before work. In that class I was also required to keep a food journal. The food journal was really just a way for us to know what was going in because hey, knowledge is power, right? Anyway, when I started that program I was sitting at around 235. I managed to drop over 15 pounds during that program but then my world changed. Or maybe it just got re-prioritized.
Starting a new job is stressful. You have people to impress, changes to adapt to, a new commute to get used to and lunches to get to know people, among other things. From that original 218 or so that I was down to I have now managed to put back on that weight and then some to put myself at a nice, plump 253 or so depending on which scale i'm standing on and how many clothes I'm wearing at the time. Let's not even talk about the BMI percentage right now.
As it came to be closer to Christmas, the inevitable question arises. What do you want for Christmas? Honestly I have everything I need and more but I still thought about it. I had read about these new fitness gadgets like the Nike Fuel Band and the Fitbit and others. After doing some research I thought that the Fitbit would work into my lifestyle nicely and provide some real data to help me begin to make the changes necessary to lose weight. On Christmas morning I opened up my new Fitbit One and also the matching Fitbit Aria Scale. The products work together with an internet based software solution called the Fitbit dashboard.
By signing up and then syncing your devices to the account you will begin to accumulate data that will help you see a clear picture of what your intake and output really is. Being the data oriented guy that I am I was well into this thing right out of the gate. The Fitbit one tracks the following:
1. Steps taken
2. Calories burned
3. Stairs Climbed
4. Sleep Quality
Some of this is based on tried and true formulas after you give some specifications about your body and the others are derived from actual activity. The typical stride in an adult is about 2.5 feet when walking which seems to be well in line with my stride so the fitbit is pretty accurate in step counting for me. You can also adjust the stride if you're shorter or taller as necessary. The stair climber seems to be directly related to changes in elevation as walking steep hills sometimes counts as stairs but when I tried using it on the stairclimber the steps did not register as stairs (although they did register as steps).
The Calories burned are based on your statistics and throughout the day, even sedentary, you're burning calories. You set a target weight loss goal (in my case 2 pounds per week) and the software will tell you how many calories you can eat for the day. As you record your food consumed, walk, climb stairs and just generally go through your daily motions the calories remaining will change. If you manage to keep your intake less than your expenditure you should lose weight. Sounds simple in theory right? So why do I need this little device to help me do it?
Well, the Fitbit is a data lover's dream! In order to sync the data you plug a small device into a usb port on your computer. Once this is done and you have the sync software installed, each time you come near it with the fitbit the data gets uploaded and is available in the Fitbit dashboard. This gives you near realtime views of how your day is progressing with respect to your weight loss goals. If you're sitting at 2:00 pm and you see you only have 500 calories left for the day you either get off your butt and go walking/stair climbing or you know that dinner is going to be a lean cuisine and maybe an ounce of almonds. If you decide to move then the activity you add will immediately give you some buffer in your calories remaining for the day.
The Fitbit also tries to be cute and helpful with a community of people to interact and compete with. Unfortunately this is where things fall off a little bit for me. Yeah I like blogging and sharing my success/failures to a degree but do I really want complete strangers checking out my BMI? If you have enough real life friends that use Fitbit maybe that would be good but so far this just seems to be a novelty. The other area that is lacking a bit is user forums. There are several thousand of them but there is not a way to search and find forums that are relative to you or your particular situation. Instead you're left to click through them page by page until you find one you like, trust me, you'll get through maybe 10 or so pages before you give up looking. If you do add a few friends you can decide what to share with them which is nice ( I only share steps taken) and then send messages or cute little taunts or praises which may help some people but will probably get old very quickly.
The best parts of the Fitbit dashboard for me are centered around the calories that I have left to consume.
Also, the graph that shows me the level of activity that I've performed for they day is helpful. This essentially tells me how often I'm being a slug. As a desk jockey this is a great reminder to get up and move around every hour or so!
There are also other areas of information that can also help you get a better overall picture of your health such as the sleep recorder. With the Fitbit you are given a wrist strap that the fitbit slips into. Once it is in and you are ready to go to sleep you press and hold down the one button which puts it into sleep mode then you just go to sleep. When you wake up (to the scheduled silent alarm if you prefer) you press and hold the button again. Once you walk by the wireless receiver the data will be uploaded and you'll see either how good or how bad you slept.
This is cool information to have but it doesn't really tell me how to fix it or if it even needs to be fixed. I just know I woke up a lot. I guess as I map this data out I will begin to see how I feel with less or more sleep. The most eye opening thing to me was how many times I usually wake up during the night. At this point I will also mention you can pay for a premium mebership. For $49.95 you get access to additional graphs and logs and also access to real personal trainers that go over your data and help you make positive changes to your plan. Maybe with this I would find out more about the sleep patterns.
So what else is good about the Fitbit? The battery life seems solid. The documentation says it can go for 5-7 days without a charge and then only needs 2 hours to get a full charge. So far after 2-3 days I have plugged it in and the battery has been around 3/4 full, good to go there. There is also a guy who's created a 3rd party application to notify you when it needs to be charge. You can find his page at http://fitbit.jcs.org/. The unit's form factor is very small and the belt clip that it came with seems very good with not much movement to it while walking. I did manage to turn it off accidentally once while driving, I guess my belt or pants had somehow pushed and held the button down. This was disappointing as it didn't record data until I turned it back on and there is no way to enter steps directly. You can directly log activity but you'll just enter it as a distance walk. I suppose this is good though as people could easily cheat the system and it would ruin the social competition.
The Aria scale is pretty cool. I'll admit I had some serious problems getting it to work when I took it out of the box. After some googling I found that the scale only support Wireless B. This forced me to make a change to my router to turn on support for it. Once I did this the scale worked great. Some people have also complained about its range. I have mine in the room directly above where the router is so no problems there but your mileage may vary. You can add multiple users to the scale as well which is very nice, up to 8 I think is the limit. This way people can step on it and it logs their data separately. Supposedly there is a slight inconvenience with multiple users of the same body mass as you have to select your username once stepping off the scale but that is understandable. How else would it know?
So how have my results been so far? Well, I officially started on December 27th, 2012 and today is January 4th, 2013 so I have 9 days in to date. To date I have only missed my 10,000 step goal on one day and have lost 8.1 pounds. I will admit that I'm being pretty good on my food intake. Only one day has slipped and that was New Year's Eve on which I had far too many beers to stay in range of my calorie goal. The Fitbit is something that I'm wearing regularly and looking forward to meeting my goals with every day. I have a few friends out there that I'm connected with but I'm only sharing steps with them. It is fun to see the leaderboard move up and down as we each attempt our goals. Most importantly though this device has opened up my eyes to the lack of physical activity I was getting every day as an IT professional who is bound to a desk all day. I'm not going to say it has worked yet because I've only been on the plan for 9 days but I will say it has been a revelation.
I will post periodically on my progress here, thanks for reading!
The years tend to take a toll on us if we don't continually maintain ourselves. For me, they've been particularly heavy on the pounds. So here I am at the ripe old age of 43 and staring an overage of about 75 pounds dead in the face. There are some serious health problems that can and will begin to crop up when you're pushing around that much extra weight. To be fair, like I said, I don't "feel" like the fat guy in the room most of the time but I do notice that in most cases I am probably the heaviest guy in meetings or in general gatherings with my friends. Even saying that in writing makes me know that something has to change.
I've dieted but only one time really seriously. About 2 years ago before I started my new job I was a member at the Gym where I worked and joined a bootcamp program. Three times a week I got up at 6:00 am to go exercise before work. In that class I was also required to keep a food journal. The food journal was really just a way for us to know what was going in because hey, knowledge is power, right? Anyway, when I started that program I was sitting at around 235. I managed to drop over 15 pounds during that program but then my world changed. Or maybe it just got re-prioritized.
Starting a new job is stressful. You have people to impress, changes to adapt to, a new commute to get used to and lunches to get to know people, among other things. From that original 218 or so that I was down to I have now managed to put back on that weight and then some to put myself at a nice, plump 253 or so depending on which scale i'm standing on and how many clothes I'm wearing at the time. Let's not even talk about the BMI percentage right now.
As it came to be closer to Christmas, the inevitable question arises. What do you want for Christmas? Honestly I have everything I need and more but I still thought about it. I had read about these new fitness gadgets like the Nike Fuel Band and the Fitbit and others. After doing some research I thought that the Fitbit would work into my lifestyle nicely and provide some real data to help me begin to make the changes necessary to lose weight. On Christmas morning I opened up my new Fitbit One and also the matching Fitbit Aria Scale. The products work together with an internet based software solution called the Fitbit dashboard.
By signing up and then syncing your devices to the account you will begin to accumulate data that will help you see a clear picture of what your intake and output really is. Being the data oriented guy that I am I was well into this thing right out of the gate. The Fitbit one tracks the following:
1. Steps taken
2. Calories burned
3. Stairs Climbed
4. Sleep Quality
Some of this is based on tried and true formulas after you give some specifications about your body and the others are derived from actual activity. The typical stride in an adult is about 2.5 feet when walking which seems to be well in line with my stride so the fitbit is pretty accurate in step counting for me. You can also adjust the stride if you're shorter or taller as necessary. The stair climber seems to be directly related to changes in elevation as walking steep hills sometimes counts as stairs but when I tried using it on the stairclimber the steps did not register as stairs (although they did register as steps).
The Calories burned are based on your statistics and throughout the day, even sedentary, you're burning calories. You set a target weight loss goal (in my case 2 pounds per week) and the software will tell you how many calories you can eat for the day. As you record your food consumed, walk, climb stairs and just generally go through your daily motions the calories remaining will change. If you manage to keep your intake less than your expenditure you should lose weight. Sounds simple in theory right? So why do I need this little device to help me do it?
Well, the Fitbit is a data lover's dream! In order to sync the data you plug a small device into a usb port on your computer. Once this is done and you have the sync software installed, each time you come near it with the fitbit the data gets uploaded and is available in the Fitbit dashboard. This gives you near realtime views of how your day is progressing with respect to your weight loss goals. If you're sitting at 2:00 pm and you see you only have 500 calories left for the day you either get off your butt and go walking/stair climbing or you know that dinner is going to be a lean cuisine and maybe an ounce of almonds. If you decide to move then the activity you add will immediately give you some buffer in your calories remaining for the day.
The Fitbit also tries to be cute and helpful with a community of people to interact and compete with. Unfortunately this is where things fall off a little bit for me. Yeah I like blogging and sharing my success/failures to a degree but do I really want complete strangers checking out my BMI? If you have enough real life friends that use Fitbit maybe that would be good but so far this just seems to be a novelty. The other area that is lacking a bit is user forums. There are several thousand of them but there is not a way to search and find forums that are relative to you or your particular situation. Instead you're left to click through them page by page until you find one you like, trust me, you'll get through maybe 10 or so pages before you give up looking. If you do add a few friends you can decide what to share with them which is nice ( I only share steps taken) and then send messages or cute little taunts or praises which may help some people but will probably get old very quickly.
The best parts of the Fitbit dashboard for me are centered around the calories that I have left to consume.
Also, the graph that shows me the level of activity that I've performed for they day is helpful. This essentially tells me how often I'm being a slug. As a desk jockey this is a great reminder to get up and move around every hour or so!
There are also other areas of information that can also help you get a better overall picture of your health such as the sleep recorder. With the Fitbit you are given a wrist strap that the fitbit slips into. Once it is in and you are ready to go to sleep you press and hold down the one button which puts it into sleep mode then you just go to sleep. When you wake up (to the scheduled silent alarm if you prefer) you press and hold the button again. Once you walk by the wireless receiver the data will be uploaded and you'll see either how good or how bad you slept.
This is cool information to have but it doesn't really tell me how to fix it or if it even needs to be fixed. I just know I woke up a lot. I guess as I map this data out I will begin to see how I feel with less or more sleep. The most eye opening thing to me was how many times I usually wake up during the night. At this point I will also mention you can pay for a premium mebership. For $49.95 you get access to additional graphs and logs and also access to real personal trainers that go over your data and help you make positive changes to your plan. Maybe with this I would find out more about the sleep patterns.
So what else is good about the Fitbit? The battery life seems solid. The documentation says it can go for 5-7 days without a charge and then only needs 2 hours to get a full charge. So far after 2-3 days I have plugged it in and the battery has been around 3/4 full, good to go there. There is also a guy who's created a 3rd party application to notify you when it needs to be charge. You can find his page at http://fitbit.jcs.org/. The unit's form factor is very small and the belt clip that it came with seems very good with not much movement to it while walking. I did manage to turn it off accidentally once while driving, I guess my belt or pants had somehow pushed and held the button down. This was disappointing as it didn't record data until I turned it back on and there is no way to enter steps directly. You can directly log activity but you'll just enter it as a distance walk. I suppose this is good though as people could easily cheat the system and it would ruin the social competition.
The Aria scale is pretty cool. I'll admit I had some serious problems getting it to work when I took it out of the box. After some googling I found that the scale only support Wireless B. This forced me to make a change to my router to turn on support for it. Once I did this the scale worked great. Some people have also complained about its range. I have mine in the room directly above where the router is so no problems there but your mileage may vary. You can add multiple users to the scale as well which is very nice, up to 8 I think is the limit. This way people can step on it and it logs their data separately. Supposedly there is a slight inconvenience with multiple users of the same body mass as you have to select your username once stepping off the scale but that is understandable. How else would it know?
So how have my results been so far? Well, I officially started on December 27th, 2012 and today is January 4th, 2013 so I have 9 days in to date. To date I have only missed my 10,000 step goal on one day and have lost 8.1 pounds. I will admit that I'm being pretty good on my food intake. Only one day has slipped and that was New Year's Eve on which I had far too many beers to stay in range of my calorie goal. The Fitbit is something that I'm wearing regularly and looking forward to meeting my goals with every day. I have a few friends out there that I'm connected with but I'm only sharing steps with them. It is fun to see the leaderboard move up and down as we each attempt our goals. Most importantly though this device has opened up my eyes to the lack of physical activity I was getting every day as an IT professional who is bound to a desk all day. I'm not going to say it has worked yet because I've only been on the plan for 9 days but I will say it has been a revelation.
I will post periodically on my progress here, thanks for reading!
Wednesday, September 19, 2012
My essential Android apps list
Over time and through two full cycles of Android phones (The HTC Incredible and now the Samsung Galaxy S3) I have come to rely on quite a few applications. It was interesting to me when I picked up my new phone in July because I really got a chance to reconfigure my phone and the various home screens. Since then I've even dumped a few apps and found a few new ones but these are the staples of my Android life and I hope that you will check them out when you have time. Each of these apps I use on a regular basis. I have a lot more on my phone but this is the list that would go with me to my next phone and beyond. Just hit the Google Play link and type in the app's name to find it. Most of them are free or low cost so what are you waiting for, get out there and find some new stuff to play with on your phone!
Google Play Store
Everyday Use
Dropbox - Instant camera uploads to the cloud and your other files always available
Weatherbug HD - Great forecasts and a cool widget for your homescreen
Dolphin Browser - The best alternate browser hands down
Flixster - Find movie times and watch trailers
Gas Buddy - Find the cheapest gas!
RadarNow - Shows instant radar wherever you are, great for camping and hiking
Google Reader - Catch up on all your rss feeds
Springpad - Take notes on anything, I use it for grocery lists household notes like filter sizes
Shop Savvy - Scan a UPC code get the cheapest price around
QuickOffice - View excel, word and powerpoint documents on your phone
Craigslist - Post items for sale easily, browse what's available to buy
Key Ring - Store all those loyalty cards on your phone
Slacker Radio - Great mobile radio with good stations to choose from
Netflix - Watch shows while working out or anytime
Flashlight - Light the way, widget provides instant on and off from home screen
Mint - Track our finances
Redbox - Find out if your local Red Box has that movie you want for tonight
Utilities
Speed Test - Shows how fast your internet connection is
Elixer - Flexible extensions to android such as more icons and instant tether on and off
Games
Cut the Rope - Great puzzle game
FlickGolf! - Relaxing and fun sports game
Hex Defense - Challenging real time tower defense
Jetpack Joyride - Fun arcade challenge
Plants Vs Zombies - A great classic tower defense game
Skateboard - Fun arcade action with great unlocks
Where's my Water - Interesting puzzle game that will make you think
Wind-up Knight - Platformer with cool graphics
Other Apps
CamScanner - Scan any document into a pdf, receipts, etc.
Shazam - Listens for playing music and will tell you what song/artist is
NFL Mobile - Track your favorite team and all scores
Yahoo Fantasy Football - Keep up with your fantasy team
WiFi Analyzer - Shows you signal strength of available wifi
Zillow - Real Estate pricing
Google Play Store
Everyday Use
Dropbox - Instant camera uploads to the cloud and your other files always available
Weatherbug HD - Great forecasts and a cool widget for your homescreen
Dolphin Browser - The best alternate browser hands down
Flixster - Find movie times and watch trailers
Gas Buddy - Find the cheapest gas!
RadarNow - Shows instant radar wherever you are, great for camping and hiking
Google Reader - Catch up on all your rss feeds
Springpad - Take notes on anything, I use it for grocery lists household notes like filter sizes
Shop Savvy - Scan a UPC code get the cheapest price around
QuickOffice - View excel, word and powerpoint documents on your phone
Craigslist - Post items for sale easily, browse what's available to buy
Key Ring - Store all those loyalty cards on your phone
Slacker Radio - Great mobile radio with good stations to choose from
Netflix - Watch shows while working out or anytime
Flashlight - Light the way, widget provides instant on and off from home screen
Mint - Track our finances
Redbox - Find out if your local Red Box has that movie you want for tonight
Utilities
Speed Test - Shows how fast your internet connection is
Elixer - Flexible extensions to android such as more icons and instant tether on and off
Games
Cut the Rope - Great puzzle game
FlickGolf! - Relaxing and fun sports game
Hex Defense - Challenging real time tower defense
Jetpack Joyride - Fun arcade challenge
Plants Vs Zombies - A great classic tower defense game
Skateboard - Fun arcade action with great unlocks
Where's my Water - Interesting puzzle game that will make you think
Wind-up Knight - Platformer with cool graphics
Other Apps
CamScanner - Scan any document into a pdf, receipts, etc.
Shazam - Listens for playing music and will tell you what song/artist is
NFL Mobile - Track your favorite team and all scores
Yahoo Fantasy Football - Keep up with your fantasy team
WiFi Analyzer - Shows you signal strength of available wifi
Zillow - Real Estate pricing
Monday, May 21, 2012
DBMSaaS - Database Management System as a Service
DBMSaaS – Database Management System as a Service
Bill Schoonmaker - Vice President of Data Architecture at Adaptivity, Inc
Bill Schoonmaker - Vice President of Data Architecture at Adaptivity, Inc
May 10th, 2012
Consider cost savings, risk reduction and space
savings. While these may not be the
first things you think of when you think of your database, all are reasons to discover
that recently, a new paradigm shift has begun.
While there seems to be some disagreement on a proper acronym or even
name, for this blog, I will refer to it as DBMSaaS, or Database Management
System as a Service. This is different
than DaaS (Data as a Service) as that is typically used for data sets that can
be used across various lines of business, usually slowly changing lookup data. A good example of DaaS would be the Microsoft Azure Markeplace. There you’ll find subscription ready data to
access from your applications. You don’t
have to manage or maintain the data, you just use it. Now that we understand the differences
between DaaS and DBMSaaS, why exactly would someone be interested in putting
their database in the cloud?
In today’s cloud centric world, there are many vendors that
are offering infrastructure as a service (IaaS). Vendors such as Amazon, Microsoft, 6 Fusion,
Rackspace and dozens more provide the ability to quickly stand up all types of
virtual servers. Typically if you are
using an IaaS vendor you will deploy a virtual server for your database in
addition to your application and web servers.
Your database software is then installed on the server and the databases
you need are configured and maintained.
This is similar to what would be done in your own environment only you
are not housing the server on your own hardware.
With DBMSaaS, there is no need for you to manage the server or
in some cases even manage the database.
You simply set up the structure of your database and load the data. Backups and database upgrades are handled by
the vendor and in some cases horizontal scalability is as easy as clicking a few
buttons or may be automatic. Microsoft
SQL Azure, Amazon Relational Database
Service and Oracle Database
Cloud Service are just three of the ready to deploy solutions for setting
up and maintaining your database in the cloud.
In addition to these three, there are a growing number of DBMSaaS providers. You should have no trouble finding one that
suits your needs. But, outside of vendor
selection, how do you make a choice to go to the cloud with your database at
all?
There are a few important questions you will need to answer first
about your existing database before you can consider the move to DBMSaaS. Questions such as, how secure does my data
need to be? Or, how much data do I need
to store in my database? And, what kind
of throughput do I need for my database?
Addressing these questions up front will help you begin to understand if
your database is right for cloud deployment.
If you need absolute control over your data and manage access on a user
by user basis then your database is probably better off in your own environment
behind your own firewall. If you need to
store more than a terabyte of data, you’re more than likely better serviced by using
your own infrastructure. Finally, if you
need extremely fast (sub second) access to your data and most of your access
happens on your own network then you are probably better off rolling your own.
The next logical step is find support for your particular
DBMS. What database engine does it run
on? There are solutions for Oracle, SQL
Server, MySQL, CouchDB and a few others.
If your database currently runs on Sybase ASE for example, you would not
be a candidate for DBMSaaS as there isn’t vendor support for it yet. Sybase is working with Amazon RDS to
implement their DBMS in the Cloud but no release date has been specified. Similarly, if you are running DB2 then there
currently isn’t an option either. If you
find that your DBMS is not supported by a DBMSaaS vendor then you will either
be tackling a DBMS migration project or considering traditional IaaS.
Now, you’ve made it to the next step. There is a vendor that supports your DBMS and
you’ve answered some high level questions about your database and its needs. Now what?
There are myriad additional questions that should be asked before you
make the leap to DBMSaaS. These
questions range from low level database features such as high availability,
backup operations and encryption all the way to business intelligence support. Each
vendor offers their own feature set so you will need to explore each one in
order to find out what is best for you.
In general, pricing is competitive.
Especially when you factor in the cost of a full-time database
administrator and the license for the database itself these services begin to
make a lot of sense. That is, IF your
database meets the right criteria. The
vendors will continue to enhance their DBMSaaS offerings. Staying informed through the various blogs
and product announcements may mean that an answer of no today may be an answer
of yes in the future. The right partner
can help you make intelligent decisions.
Adaptivity prides itself on our ability to help you make the right
decisions with regard to cloud suitability for your applications and
databases. Check out our offerings today
at www.adaptivity.com.
Tuesday, April 24, 2012
iPhone Adventures
I finally moved my wife Suzi out of the phone stone ages. She was sporting a Samsung Intensity II phone for the last 2 years. Her primary complaint over smart phones was that she literally HATED touchscreens. I tried many times to get her to play around with my Android phone but she never could quite the hang of it. I knew that when it came time to update her phone that we would be going with the iPhone.
The iPhone is approachable by just about anyone. I would venture to guess that if you put an iPhone and an Android side by side and turned them on to their respective ready modes that the majority of people would gravitate towards the Apple product. Why? I do not know. What I do know, and this is from experience, the Android platform can be quite frustrating. Yes, you can spend some time setting it up to look like the iPhone and act like an iPhone but in the end, it is not an iPhone. The platform is getting better but I believe it is for more of a power user.
Even the iPhone takes some getting used to. Understanding what all the applications are for and getting around the user interface are not entirely intuitive but they can be quickly learned. My largest stumbling blocks with setting things up for her came in two flavors.
1. Getting her Email to properly sync across her pc based outlook and phone
2. Understanding Home Sharing so we could share purchased apps
Email turned out to be a pretty large hassle and took several hours to rectify. Suffice to say that using the IMAP protocol over POP3 is the way to go. With Suzi only ever checking email on her PC Based Outlook client I had originally set her up to use POP3. There were PST files stored locally on her hard drive so when she checked email the messages were downloaded off the server but not deleted. Outlook was smart enough to know if a message was already stored locally so it was never an issue. Setting up email on the iphone though meant that in her case, her inbox was flooded with over 800 messages while her local inbox had exactly 9!
In order to change her over to IMAP I had to create a new mailbox on her outlook client then manually move the messages over from the POP3 account to the IMAP account that she wanted to keep. This was very time consuming as she's been using the same email address for years. Basically everything has to be uploaded to the server. Since I recently switched over Gmail as my email provider for schoonsplace.com it was good that they provide 8 GB of storage for email, this was not something I had thought of before leaving Host Monster for email services as they were virtually unlimited for space. I had some availability issues with them though so I switched over to Gmail.
Once I got everything switched over the mail has the proper behavior now. Messages will be visible in both inboxes but if you delete from one it is deleted from the other. Essentially the clients are reading everything off the server so it just works. About the only downsides I see are speed and space. The actual send/receive process does take longer and as I mentioned above I now have the 8GB limit. It's not a huge deal though as the send/receive is maybe 1 second longer and she's only using 1/8th of the limit so I suspect we'll be fine. I can always help her to copy large emails over to the local PST files on her laptop if she wants to keep them or just get her into the habit of downloading attachment emails and then deleting them.
Home sharing was another animal altogether. I must admit that I should have done my homework on this one upfront because I think I made a couple of rookie mistakes. One was that I used the same email address on her iPhone that I use for our iPad 2. This is not necessary as with home sharing as long as you are on the WiFI network you have access to the other email address's library. The only thing you lose is the ability to actually install apps from the phone itself that were purchased using the other library's login. It was a bit confusing at first but in the end it all seems to work well. The other issue I ran into was when the PC that I use for itunes synching blanks the monitor (doesn't go to sleep just shuts off the monitor) for some reason the home sharing stops working. I can still see the library but I can't actually transfer any content. Not sure if this is the way it was intended or a bug.
I will also mention that we just went with an iPhone 4. I know the 5 is coming soon and the S has Siri but really, for a person going to their first smart phone the iPhone 4 is plenty of power. If she becomes a power user then we can look at upgrading her in the next cycle with Verizon. By then I can only imagine the phones that will be available.
The iPhone is approachable by just about anyone. I would venture to guess that if you put an iPhone and an Android side by side and turned them on to their respective ready modes that the majority of people would gravitate towards the Apple product. Why? I do not know. What I do know, and this is from experience, the Android platform can be quite frustrating. Yes, you can spend some time setting it up to look like the iPhone and act like an iPhone but in the end, it is not an iPhone. The platform is getting better but I believe it is for more of a power user.
Even the iPhone takes some getting used to. Understanding what all the applications are for and getting around the user interface are not entirely intuitive but they can be quickly learned. My largest stumbling blocks with setting things up for her came in two flavors.
1. Getting her Email to properly sync across her pc based outlook and phone
2. Understanding Home Sharing so we could share purchased apps
Email turned out to be a pretty large hassle and took several hours to rectify. Suffice to say that using the IMAP protocol over POP3 is the way to go. With Suzi only ever checking email on her PC Based Outlook client I had originally set her up to use POP3. There were PST files stored locally on her hard drive so when she checked email the messages were downloaded off the server but not deleted. Outlook was smart enough to know if a message was already stored locally so it was never an issue. Setting up email on the iphone though meant that in her case, her inbox was flooded with over 800 messages while her local inbox had exactly 9!
In order to change her over to IMAP I had to create a new mailbox on her outlook client then manually move the messages over from the POP3 account to the IMAP account that she wanted to keep. This was very time consuming as she's been using the same email address for years. Basically everything has to be uploaded to the server. Since I recently switched over Gmail as my email provider for schoonsplace.com it was good that they provide 8 GB of storage for email, this was not something I had thought of before leaving Host Monster for email services as they were virtually unlimited for space. I had some availability issues with them though so I switched over to Gmail.
Once I got everything switched over the mail has the proper behavior now. Messages will be visible in both inboxes but if you delete from one it is deleted from the other. Essentially the clients are reading everything off the server so it just works. About the only downsides I see are speed and space. The actual send/receive process does take longer and as I mentioned above I now have the 8GB limit. It's not a huge deal though as the send/receive is maybe 1 second longer and she's only using 1/8th of the limit so I suspect we'll be fine. I can always help her to copy large emails over to the local PST files on her laptop if she wants to keep them or just get her into the habit of downloading attachment emails and then deleting them.
Home sharing was another animal altogether. I must admit that I should have done my homework on this one upfront because I think I made a couple of rookie mistakes. One was that I used the same email address on her iPhone that I use for our iPad 2. This is not necessary as with home sharing as long as you are on the WiFI network you have access to the other email address's library. The only thing you lose is the ability to actually install apps from the phone itself that were purchased using the other library's login. It was a bit confusing at first but in the end it all seems to work well. The other issue I ran into was when the PC that I use for itunes synching blanks the monitor (doesn't go to sleep just shuts off the monitor) for some reason the home sharing stops working. I can still see the library but I can't actually transfer any content. Not sure if this is the way it was intended or a bug.
I will also mention that we just went with an iPhone 4. I know the 5 is coming soon and the S has Siri but really, for a person going to their first smart phone the iPhone 4 is plenty of power. If she becomes a power user then we can look at upgrading her in the next cycle with Verizon. By then I can only imagine the phones that will be available.
Wednesday, April 11, 2012
Which smart phone to choose?
So these days every 22 months or so I start to get excited about phone again. I wish I had written down somewhere all of the mobile phones I've owned over the years. My first mobile phone was actually a bag phone and I remember making a call from some beach down in Florida back to my office. It wasn't even really a necessary call, it was a bragging call. "Hey Boss, how's it going? Guess where I am". Anyway, I'm sure it costs me at least a few dollars to make that call and I'm even more sure I looked like a total geek dragging that black bag around with me everywhere as if I HAD to be always connected. We've come a long ways since then for sure. I've had my share of flip phones that were the ancestors to today's smart phones. They included calculators and calendars and contact lists but didn't have the ability to side-load apps from anywhere easy.
I've been on the Verizon network now for probably at least 10 years. My current hand set is the HTC Incredible. I've had my share of problems with it but my most recent factory refresh seems to have made most of those go away. I purchased the Incredible almost sight unseen on a recommendation from a friend. Being a tech geek this was a bit odd for me as I usually do my own due diligence for the tech I buy. Most certainly for something I'll carry with me everywhere I go for two years. So this time, it had to be different. I should mention that I've also dabbled in the Apple world but never with the iPhone. I own both a 1st generation iPod Touch and an iPad 2. I have enjoyed both devices although the iPod Touch is now mostly a paperweight. The iPad 2 however, is the jewel of the family gadgets. Between myself and the 6 year old it's gotten quite a bit of usage and I'm heavily invested in applications. This alone is enough to make me consider an iPhone for this contract cycle.
So what's making me not want to pull the trigger? With Android, I know I will have flexibility. With Android, I know I will be able to make my home screens look and feel the way I want to. I am a big widget fan, Of my 7 screens in HTC Sense only 2 of them don't contain widgets. I have widgets for google search, for volume control, for YouTube, for Slacker Radio, for device control and my Twitter feed. They are all useful and make my phone that much better and easy to use without having to launch into apps all the time just to figure out what is going on. With the iPhone, I haven't seen widgets, about the closest thing I can find is shortcuts but these really only take you the settings area where you still perform the action. One of my favorite widgets on Android is the Instant Camera, with it on my home screen one press and I hold the camera up, it shoots up to the 3 shots then uploads them to Dropbox, pretty cool huh?
So where does that leave me the iPhone. Siri and the retina display? Ok, let me back up a second, I haven't even really said what phones I am looking at. I have narrowed my choices down to the iPhone 4S 32GB and the Motorola Droid Razor Maxx. The Samsung Nexus was in the running but the reports on battery left left a bad taste in my mouth. I had already purchased a larger battery for my incredible just a month or so after buying it and the battery life is still not great on it. With the Maxx's battery I can do everything longer than either the iPhone 4S or the Nexus according to this propaganda sheet . Seriously, FOURTEEN hours of web surfing, I don't even think I could do 4 on my Incredible, and that's only 3G. Anyway, if battery power is my number one concern then that may seal the deal. Let's look at the list of features in order of want
1. Battery Life
2. Camera Shutter Speed/Quality
3. Apps Selection
4. Screen Visibility Outdoors
5. Screen Size
Battery life, the Maxx wins hands down in this category. With regard to camera, most sites give it to the 4S over the Maxx, but, the Maxx has a flash. I use this feature constantly and one of the really cool features of my current Incredible is being able to use the flash as an effective flash light, I have done this many, many times and even have an on/off widget on my home screen for the flash light. With apps selection comes a tough decision. I haven't spent nearly as much money on apps for the android platform as I have with the IOS. With all of these apps, the moment I bring the 4S into the house I can sync all these apps over to it that are compatible. This would be a tremendous cost saver BUT I already have all the apps I need on my Incredible, they will also port to the new Maxx if I choose that phone. I think this one is a wash really. People seem torn on screen visibility in direct sunlight and to be honest, I work in a cube so this one could be argued as a moot point. Finally, screen size, as I get older I find myself holding the phone farther away to read it and additional real estate would be welcome. The Maxx wins in this category.
So, I think what it boils down to here is I just want the Maxx. I honestly feel like i'm just settling if I go with the iPhone 4S. I will be purchasing the iPhone for my wife though as she needs the ease of use. Me, I think I just want to continue with the Android platform. I did consider the Windows Phone but I am just afraid for lack of support there. The newest Nokia handsets are looking good on that platform and people really seem to like them but I am too used to Android to go that route just now.
Maybe next time Microsoft. Now, to figure out where I'll get the best deal. Seems Best Buy had a $50 gift card thing going on a few months back that I signed up for which will bring the price down to an easier to digest $250. I will be sure to post more insights into the phone once I pick it up hopefully this weekend or next. Thanks for reading.
I've been on the Verizon network now for probably at least 10 years. My current hand set is the HTC Incredible. I've had my share of problems with it but my most recent factory refresh seems to have made most of those go away. I purchased the Incredible almost sight unseen on a recommendation from a friend. Being a tech geek this was a bit odd for me as I usually do my own due diligence for the tech I buy. Most certainly for something I'll carry with me everywhere I go for two years. So this time, it had to be different. I should mention that I've also dabbled in the Apple world but never with the iPhone. I own both a 1st generation iPod Touch and an iPad 2. I have enjoyed both devices although the iPod Touch is now mostly a paperweight. The iPad 2 however, is the jewel of the family gadgets. Between myself and the 6 year old it's gotten quite a bit of usage and I'm heavily invested in applications. This alone is enough to make me consider an iPhone for this contract cycle.
So what's making me not want to pull the trigger? With Android, I know I will have flexibility. With Android, I know I will be able to make my home screens look and feel the way I want to. I am a big widget fan, Of my 7 screens in HTC Sense only 2 of them don't contain widgets. I have widgets for google search, for volume control, for YouTube, for Slacker Radio, for device control and my Twitter feed. They are all useful and make my phone that much better and easy to use without having to launch into apps all the time just to figure out what is going on. With the iPhone, I haven't seen widgets, about the closest thing I can find is shortcuts but these really only take you the settings area where you still perform the action. One of my favorite widgets on Android is the Instant Camera, with it on my home screen one press and I hold the camera up, it shoots up to the 3 shots then uploads them to Dropbox, pretty cool huh?
So where does that leave me the iPhone. Siri and the retina display? Ok, let me back up a second, I haven't even really said what phones I am looking at. I have narrowed my choices down to the iPhone 4S 32GB and the Motorola Droid Razor Maxx. The Samsung Nexus was in the running but the reports on battery left left a bad taste in my mouth. I had already purchased a larger battery for my incredible just a month or so after buying it and the battery life is still not great on it. With the Maxx's battery I can do everything longer than either the iPhone 4S or the Nexus according to this propaganda sheet . Seriously, FOURTEEN hours of web surfing, I don't even think I could do 4 on my Incredible, and that's only 3G. Anyway, if battery power is my number one concern then that may seal the deal. Let's look at the list of features in order of want
1. Battery Life
2. Camera Shutter Speed/Quality
3. Apps Selection
4. Screen Visibility Outdoors
5. Screen Size
Battery life, the Maxx wins hands down in this category. With regard to camera, most sites give it to the 4S over the Maxx, but, the Maxx has a flash. I use this feature constantly and one of the really cool features of my current Incredible is being able to use the flash as an effective flash light, I have done this many, many times and even have an on/off widget on my home screen for the flash light. With apps selection comes a tough decision. I haven't spent nearly as much money on apps for the android platform as I have with the IOS. With all of these apps, the moment I bring the 4S into the house I can sync all these apps over to it that are compatible. This would be a tremendous cost saver BUT I already have all the apps I need on my Incredible, they will also port to the new Maxx if I choose that phone. I think this one is a wash really. People seem torn on screen visibility in direct sunlight and to be honest, I work in a cube so this one could be argued as a moot point. Finally, screen size, as I get older I find myself holding the phone farther away to read it and additional real estate would be welcome. The Maxx wins in this category.
So, I think what it boils down to here is I just want the Maxx. I honestly feel like i'm just settling if I go with the iPhone 4S. I will be purchasing the iPhone for my wife though as she needs the ease of use. Me, I think I just want to continue with the Android platform. I did consider the Windows Phone but I am just afraid for lack of support there. The newest Nokia handsets are looking good on that platform and people really seem to like them but I am too used to Android to go that route just now.
Maybe next time Microsoft. Now, to figure out where I'll get the best deal. Seems Best Buy had a $50 gift card thing going on a few months back that I signed up for which will bring the price down to an easier to digest $250. I will be sure to post more insights into the phone once I pick it up hopefully this weekend or next. Thanks for reading.
Tuesday, March 1, 2011
The Death of the Personal Web Site?
So raise your hand if you still have a personal web site. (My Hand goes up). I'm not talking about a facebook page, I'm talking about a real honest to gosh domain and some interesting combination of homebrew HTML and/or a content portal such as PHP Nuke. I KNOW some of my friends have or have had them so I'm not the only one. The question is, do you still have it? You see, I received a notice from Host Monster today that my hosting package is up for renewal in 50 days. I have flip flopped with this before but this time I think I really am going to shut it down. I have this blog which to be honest is much easier to maintain and has a better editor than my PHP Nuke site and I use facebook and linkedin for pretty much everything else. The only thing I would be missing would be the photo gallery and I'm almost positive I can find a nice replacement for that. So, what do you think readers? Should I can http://www.schoonsplace.com or not?
The web site has been up for over 8 years now, actually 10 but I reinvented it after two , the first page was really straight HTML. I went to PHP Nuke and never looked back. I used Gallery for the photos and even had a few cool modules from time to time. The sad fact is, I haven't posted there in months and even when I do I usually just say that I'm not dead. I also have had issues with the gallery where I think I hit a limit or something. I can't get the Java uploader to work correctly. To renew, it's $7.95 a month for 2 years so it's around $160 total, couple that with my domain fees and the site costs me just around $200 a year. I don't want to dump the domain though as Suzi and I both use the schoonsplace.com for our personal email addresses. I found out that with EasyDNS I can get 10 email boxes for $10 extra per year so we could even keep those and not lose much if anything.
The biggest pain will be downloading the 300gb image of the site as a backup file and then figuring out what to do with it. I probably have 50% of the pictures stored locally but I'd need to go through them and figure that out, the odds of that happening are pretty slim. Then there are the comments on each picture, impossible to export and keep them intact with the actual photos, that will suck. Everything is also dated and organized by year in the gallery. Ok, so maybe that is a big pain in the butt to take them away. I just don't know, I have 50 days to decide but I will be sure to let you know whether or not the real schoonsplace.com gets shut down. Have a nice evening.
The web site has been up for over 8 years now, actually 10 but I reinvented it after two , the first page was really straight HTML. I went to PHP Nuke and never looked back. I used Gallery for the photos and even had a few cool modules from time to time. The sad fact is, I haven't posted there in months and even when I do I usually just say that I'm not dead. I also have had issues with the gallery where I think I hit a limit or something. I can't get the Java uploader to work correctly. To renew, it's $7.95 a month for 2 years so it's around $160 total, couple that with my domain fees and the site costs me just around $200 a year. I don't want to dump the domain though as Suzi and I both use the schoonsplace.com for our personal email addresses. I found out that with EasyDNS I can get 10 email boxes for $10 extra per year so we could even keep those and not lose much if anything.
The biggest pain will be downloading the 300gb image of the site as a backup file and then figuring out what to do with it. I probably have 50% of the pictures stored locally but I'd need to go through them and figure that out, the odds of that happening are pretty slim. Then there are the comments on each picture, impossible to export and keep them intact with the actual photos, that will suck. Everything is also dated and organized by year in the gallery. Ok, so maybe that is a big pain in the butt to take them away. I just don't know, I have 50 days to decide but I will be sure to let you know whether or not the real schoonsplace.com gets shut down. Have a nice evening.
Subscribe to:
Posts (Atom)