everything_is_a_test

Photo credit Fe Langdon, available on Flicker under Creative Commons.

 

Today I want to write about a specific mindset. This mindset, once adopted, can help you grow effectively. Let me first tell you how I discovered it.

The importance of a test

I have been a researcher in pattern recognition, which learns from data in an intelligent way. There are many tools available for a successful data analysis. All of them rely on certain assumptions about the data. Each assumption leads to a specific model. In the majority of the cases, however, the validity of the assumptions cannot be checked for complex data (which is often the case for real data).

I know it’s surprising, but it is true.

There are three reasons behind this. These are:

1) Too little data.

There is way too little data available for the number of unknowns in the model to be estimated

2) Algorithmic efficiency.

Even when huge data collections are at hand, a small sample of the data is used to make all the algorithms both fast and feasible for the task.

3) The lack of mathematical approaches.

No models available for in multivariate representations to check whether the given assumption holds or not.

In theory, when the assumptions about the data are true, then the best model (or one of the best) is exactly the one based on the same assumptions.

How do do you think this translates to practice?

Well …

The practice is a different story. There is usually a gap between theory and practice. You have already guessed it, right? 😉

A complex model theoretically tailored to the data distribution may loose with a seemingly irrelevant simple model. Even if this simple model is derived from a completely different assumption, it may still win with the theoretically the best model possible. It doesn’t have to be like that, of course, but it is often the case.

Why?

Because simpler assumptions lead to a few parameters. And fewer parameters can be better estimated (than the many) when there is little data.

It means that a simple model can often provide a better (though rough) fit, then a complex (hence flexible) model whose parameters are poorly estimated. This inadequate estimation often makes the complex model bad for the task.

This is a controversial point so let me paraphrase it as follows.

Imagine that a simple outline of your silhouette (aka, data) is given to a tailor (which is an algorithm). He hasn’t seen you, but he has some data about you – a rough outline of your body.

A simple model would then translate to a basic and plain dress suited around a few measurements. These may be the neck-line, the waist-line, and the chest. The dress, even though not special, will likely fit you as the basic measurements are sufficiently estimated.

A complex model would correspond to a fancy dress with layers, frills and pockets, and an asymmetric line. Many measurements are now required to have it designed well. Since they are based on your outline only, guesses have to be made. The resulting dress may look stunning, but unwearable because it would not fit. Even if beautiful, the dress may either be too narrow or too wide in wrong parts of the body, so that there is no way for you to squeeze in. 

But… If the tailor is well experienced, he is capable on choosing the right complexity based on a few measurements. He will make the right design that would be perfect for you.

The only way to know it, is to test it.

Everything is a test

In practice, when you want to guarantee the best solution for the given data you will do two things. First, you will consider a number of different models, including a variety of data transformation as well. Secondly, you will train and test them extensively on the new data. This is the data which was unused for the parameter estimation (i.e. the unknown parameters of the model) and kept aside for an evaluation.

It is a necessary step.

Without a well-designed testing stage, the primary results are often too optimistic. Moreover, your initial guesses may be totally wrong. Without rigorous testing, no intelligent solution is found. With the extensive testing and adaptation, the solution will work for new data. This is what you want.

This is a powerful learning point which easily applies to my life and your life. Namely,

[Tweet “Everything is a test.”]

What I am talking about here is the mindset, not the literal approach to test every single thing in your life. The mindset will have paramount consequences for your conscious growth.

Let me explain why.

First of all, when you approach a new idea or a habit to your life as a test, it is easy to commit when you know it is meant for your first-hand learning experience. Your goal is to see how this idea (say, a specific time management approach, weight loss program or a nutritional protocol) applies to your personal circumstances in a limited time frame. After a specified time period, you are going to evaluate how well this idea works for you.

The mindset of a tester is a mindset of a person who likes to have fun and see what happens without any specific attachment to the results. Why? Because a test is meant to provide you with feedback. When you accept that you have been just testing, it is easy to modify the approach accordingly or truly abandon it if necessary.

Secondly, if you like the newly tested idea, you choose to adopt it as your long-term habit. Even though it is now ingrained in you, after a year or two, you know you are still in a testing stage, though it is now an advanced test ;). The testing never ends. As a result, you are open to either modify it or leave it when the idea stops serving you.

Test everything

The “test everything” mindset is to enjoy running the tests, while being open to adjust them when needed. This mindset will prevent you from blindly following the gurus or getting into dogmatic thinking. A test is always subjected for an evaluation. You simply allow yourself to question both the assumptions and the results.

This mindset keeps you open for new ideas. It makes you conscious to observe when the ideas you practice have stopped serving you. It usually starts with an insight that something is a bit off track or awkward. You will notice that when your mind is set to the testing stage.

If you, however, accept the idea as an absolute truth because it comes from gurus (advanced research, your beloved one or any other authority – you name it), you may easily continue the practice it until things become so bad for you that you have no other way than connect the dots. A bit too late….

***

A tester’s mind is a versatile and flexible mind. It is a fresh mind, indeed.

As a tester, you give yourself permission to run trials of all kinds, even the ones which lead to negative results. These results are your feedback, which will be intelligently analyzed to tailor the tested approach to your specific condition. 

Choose to be a tester. You will learn a lot, adjust ideas and develop your independent thinking. Such a conscious process will teach you how to make smarter and more effective decisions.

On the top, you will get more fun!

***

 What you are going to test today?

***

 

Print Friendly, PDF & Email