In my last post a week back, I had talked about an upcoming webinar hosted by QA InfoTech on a trending topic on Application Profiling to Measure End User Experience leveraging an open source test automation solution. The event was very well received yesterday and the full video of the webinar is available below. We would love to hear any feedback and thoughts you may have on the content.

Test Automation – this has its fair share of interest, as always. This includes automation in an agile world, automation for large scale projects, skills for automated testers, managing automated test efforts etc.
  

Metrics – Bringing objectivity into the software testing effort again continues to be an area of continued interest including what is going to be relevant to measure, what not to measure in these current changing times
  

A few other core topics of ongoing interest were in areas such as: performance testing, security testing, regression testing, cloud based testing (interestingly I found fewer topics on this, this year)
  

Mobile Testing is definitely what I would call as the number one area of focus this year. There was so much interest in anything do to with mobile testing covering a range of topics such as: Mobile Testing Methodologies – Trends, Successes, Pitfalls, Mobile Applications Testing, Build your Mobile Testing Knowledge, Android Mobile Testing – Right before your Eyes, Introducing Mobile Testing to your Organization, Mobile Testing Tools 101
  

Another area of keen interest was around Crowd Sourced Testing given its potential as a great supplement to the core test techniques and its fast turnaround, cost effectiveness and overall delivery flexibility.
Stay tuned for more detailed blog posts on each of these areas in the coming months, based on what I learnt at the conference. Until then, happy testing!

Evolution of Compatibility Testing:
  

If we look at the evolution of compatibility testing, one would have started off setting up multiple physical machines each with the required system configuration and you would remember seeing compatibility testing labs filled with a bunch of machines. Those were the days when the set of browsers and operating systems were limited to a handful that could easily be tracked and tested on. Following this timeframe, were the days of virtualization software, mostly from commercial players such as Microsoft, VM Ware, CITRIX, Oracle enabling testers to test on multiple operating systems from one single machine. This still called for the tester’s time to setup and configure the machine and the virtualization software. Ongoing upgrades to the virtual copies and images, associated clean-up effort after each test pass, made this a time consuming process although it was an improvement over the earlier one on maintaining multiple physical machines. Around the same time as cloud computing hit the market, mobile computing also started booming. The combination has in some sense created a whole arena of opportunities, challenges and solutions for the testing community. Infrastructure as a Service (IaaS) has made on-demand machine and software availability so feasible saving testers’ a significant amount of setup time. From a cost angle as well, one pays for actual usage helping reduce any redundancy, wastage in obsolete machines etc. making the process both time and cost effective.
  

Current Compatibility Testing Scenario and Solutions:
  

When it comes to compatibility testing, you are mainly looking to verify your application’s functionality, UI and rendering accuracy and consistency across platforms. In the current day world, cloud computing and associated virtualization solutions on the cloud have eased the compatibility testing process significantly for the testers. A plethora of commercial and open source automated solutions are available. It is important to pick the right solution based on your requirements and constraints including – cost, time, support and customization options, parallel execution feasibility. Here is a neat consolidated list on cross browser compatibility testing including automated open source solutions available on the cloud.
  

Typically, when you are in the process of defining what your support matrix needs to be, you can rely on a couple of data points:
  

a. Look for market usage statistics such as http://www.netmarketshare.com/operating-system-market-share.aspx?qprid=8 to decide what the latest usage patterns are and optimize your test effort accordingly. This is especially useful for v1 products where you do not have a user base to start with
  

b. If you are an existing player, I would suggest you decide your test matrix, based on support that is planned for the current release, keeping in mind past usage patterns and problem areas. Google analytics kind of tools help you determine such data points that can play an important role in your decision making. Here is an article that I liked on this topic, that talks about setting up Google Analytics and helping you see usage patterns along with an array of cross browser testing tools that are available in the market today.
  

Whether you are looking at compatibility testing from a multi browser, multi device, multi platform support, each of them have their own challenges, yet are inevitable if you want your product/service to succeed in the market place. The good thing is, alongside these challenges arise new opportunities and solutions as discussed above. Clearly understanding your needs upfront, along with analysis of available solutions to hone in on what works the best for your product is an important step in finalizing a robust test strategy.
  

As we start off, one has to understand that global development centers are typically started years after the company has established itself at its home base, its headquarters. So, one has to understand that inherent challenges around disparity in domain knowledge, work process and methodologies and overall team maturity will exist. Once you acknowledge these as ground realities that need to be worked on, it becomes a more manageable task that we can objectively work towards fixing, rather than working in a defensive modus operandi. Over the rest of this write up, you will see me using global or remote employees/engineers to refer to people who work in the global delivery centers and employees at the central location for those that work in the head quarters.
  

So, let’s look at the challenges individually and the possible mitigation strategies:
  

1. Domain Knowledge – unless this is a completely new product line which is started off in the remote delivery location, typically a company’s headquarters does have a large involvement / presence in the product that is being developed. One has to understand that people at the central location are at a domain advantage due to several reasons – existing product knowledge, access to business and marketing teams, proximity to real users (unless the product has a local demand that the global center would have proximity to) etc. Domain knowledge takes time to build. While typically one may consider cross team trips to impart such knowledge, setting aside some time for building domain expertise, along with ongoing deliverables is important. While estimating the team’s productivity numbers, factor for such domain know-how gaps and chalk out a clear plan to fix those gaps within a reasonable period of time. Evaluate ongoing progress and importantly schedule reverse knowledge sessions, where you encourage your global teams to take charge of any demos, presentations etc. to be made. This not only challenges them on the job, but also motivates and gives them the right opportunities towards bridging the domain gap.
  

2. Technical Depth – if the right hiring practices have been adopted, this should not be a major challenge to handle, because the characteristics of a global delivery model do not hinder technical depth adversely a whole lot. That said, employees in the central location might be at a slight advantage due to technical discussions, trainings, conferences that they may be part of, which their counterparts in global centers may not be involved with. To mitigate these, the leadership has to plan for such trainings (if live is not possible, at least recorded sessions), virtual discussions, relevant conferences in their location to promote technical learning that is consistent across the board. It might be beneficial to invite engineers from global centers to present in international conferences, or even conferences in the central location. This helps promote technical excellence as well as motivates them to rise to the levels of their central counterparts.
  

3. Disciplinary Practices – These next three points in my opinion go a long way in establishing delivery consistency. What I mean by disciplinary practices is that every discipline – be it design, marketing, development, test etc. have their own implementation practices specific to the discipline and the company. This includes both tactical day to day practices as well as strategic futuristic practices including best practices and tracking IT trends to adopt them in their organization. Typically, the central location of the company tends to be the hub or nucleus that builds stability around disciplinary practices. This can be easily imparted to global teams over time through cross training sessions, peer reviews of deliverables (such as artifacts, code, tests etc.), involving the global teams in the overall disciplinary implementation process including taking their feedback.
  

4. Process Knowledge – When I talk about process knowledge, I specially talk about the processes adopted in the specific product team. In the previous point on the disciplinary practices, we talked about the disciplinary depth, whereas here it is about the process across the team, running horizontally across disciplines. This includes the development methodology used, the implementation strategies in place, escalation routes, reporting mechanisms, responsibilities of various team members etc. A common understanding of the processes goes a long way in establishing delivery consistency across the board. To enable this have periodic check points to see what is going well, what can be improved on etc. not just at a product and team level but also at a center specific level to catch up on any deltas that may exist
  

5. Communication Skills – Communication is often the key in making or breaking the delivery model. Even if careful planning and execution is in place on the above areas, if the right levels of communication and overall transparency have not been established, the model will soon become more of a theory than a practice and will not be able to sustain long. This is where the right leadership is very important. The leadership should promote transparency and establish a management and communication style that empowers the teams to collaborate bringing out their bests. Thanks to evolving technology, that communication challenges that existed even say a few years back are a thing of the past. Video, audio conferences, virtual meetings, webinars have all become possible at a very low or even no cost basis. Leverage communication in its varied forms and levels to understand what inconsistencies may exist and work towards addressing them in a planned manner. You will soon realize that it is no longer an overhead, rather an ammunition in building a global delivery model that is consistent across the board.
  

It is no denial that inconsistencies in delivery will exist when a global delivery center is spun off. The points mentioned above are important areas to acknowledge and address in bridging the inconsistency over time. I am sure there are several more areas to work on specific to every company and team, but the above list should definitely set a team up in the right path to promoting a smooth and consistent global delivery model.
  

In my opinion, mobility is more of a horizontal that stretches across domains and has common QA practices and methodologies that can be re-used. When you are testing for mobile applications (be it any domain – educational, banking, retail, transport etc.) or mobile powered versions of applications, it is really a horizontal like how you would test for performance or security of the application. Even if a product is solely being released on mobile platforms, it is still to be seen only as a horizontal, because the product by itself may belong to a certain domain. E.g. in the recent times, we have been testing for a travel application, which is being considered for launch as a mobile application to start off with. So, herein the domain really is travel and the mobile rendering is a horizontal. Whether you test for the application specific to the device or specific to the mobile platform, there are commonalities that you can identify to carry over across domains.
  

So, when does mobility really become a domain? In my opinion again, only when a mobile workflow centric development takes place, do I see it leaning towards a domain of its own. For e.g. when an m-commerce application is being developed,, although the domain may still be retail, a lot of workflows including login, any other application interaction, mobile payment instruments become very specific to the device or platform. Those are potential cases, where I see it being looked at, as a domain of its own.
  

As we sometimes say, what is in a name – similarly, what is in a classification? Whether a domain/vertical or a horizontal, implementing the following high level points, will go a long way in a robust mobile test strategy:
  

1. Optimized test matrix – with the multitude of devices and platforms in the mix, mobile testing can soon become overwhelming and unless a methodology is devised for optimizing the test matrix, the efforts would not reap much return. Keep track of device and platform market shares based on geographies in which the product is to be launched, past problematic areas, available time and cost budgets in choosing an optimized set to work with
  

2. Real devices to work with – while simulators and emulators help bring in at least partial test coverage, they do not completely account for real device testing. Gradually yet steadily build a real device mobile test lab, handpicking devices to include based on their market share. Look for creative ways to augment device coverage by even using some of your company employees’ phones for testing purposes, with their approval
  

3. Smart devices application test automation – this is still an evolving area, but much progress has been made both with commercial tools and open source tools that now support mobile test automation. Some require the application’s source code, while some work off of the UI objects. While the latter in not always robust, it offers automation potential in the mobile space. The automation strategy also depends on whether the application under test is native, web or hybrid. For further details on this subject, please refer to a webinar that we recently hosted on this topic
  

4. Reusable test cases – as a generic best practice, it helps to build reusable test cases for mobile testing. Tests around install, login, any local downloads, user preferences storage etc. can be easily saved for future re-use with minimal customization, even across domains. Also, build test cases around existing laid down guidelines such as HIG – Human Interface Guidelines for Android, iOS. These will help you get comprehensive coverage and also stay up-to-date, especially when you know the developer has used such guidelines in developing the application
  

5. Ongoing research – Mobile development is still in its nascent stages. There is a lot of potential and market penetration yet to happen. Look for opportunities for ongoing research specific to problems you are solving in your work, conferences that focus on mobile testing solutions, white papers, demos, webinars etc. to build the knowledge base
  

In summary, it is very exciting to witness the current ongoing mobile revolution. Embracing it as part of your testing strategy and being on the lookout for newer trends and test techniques in this space, along lines of points listed above, will help you get a leading edge in your mobile testing efforts!