I really enjoy writing automated tests(yes, really) for basically anything i am working on, i sometimes go test-first if something is clear how it’s going to end or at least clear enough
to see a possible outcome as you go.But what i don’t do is mocking!(sometimes unavoidable, but still) I usually use a set of fixtures with real data(at least as real as a predefined piece of information can be), letting my tests hit real endpoints and databases and actually thinning the barrier between unit and integration testing.

So, i had a thought, what if i could have my program run its tests using the actual real data, inputted from real users in a real state environment? How would that turn out? Would that be
feasible(ignoring runtime overhead and implementation details) and actually useful? I am still not sure but i went ahead and hacked an implementation in order to explore it.
Github

Note: in the following section i’ll use actual code with LiveUnit but i am interested in the idea, the theory if you will, not the details.

For a simple program like :

require 'liveunit'

class Example
  include LiveUnit
  def initialize
    evaluate_me #!
    @myvar = 15
  end
  def calculate(num)
    num + 10
  end
end

ex = Example.new
puts ex.calculate(40)

A live test could look like :

require 'liveunit/testcase'

class TestExample < LiveUnit::TestCase
  #doomed to fail
  def test_calculate
    msg "Result should be greater than 400 when @myvar==15"
    assert_equal(15, state[:@myvar])
    assert_operator myreturn, :>=, 400
  end
end

The test_calculate will run automatically after the calculate method completes and will notify us of a failure.
Now let’s see what our test say, pretending we see something like that in the real world :
“I should return something > 400 when my instance’s @myvar == 15”.
Imagine @myvar was @has_access == true|false and the method returns the result of a record editing operation.Could that test spot a loophole, an edge case in our program? Yes.

A normal unit test would have of course spotted this as well but maybe we would have mocked @has_access instead of letting our login system set the session variable in a real live case.

I can see the appeal to that, testing every possible edge case against every possible global state can be hard.Imagine deploying your application to a development server, turn live-testing on
and use it for a while like a normal user would and then examine the report for any fails.

###Conclusion In a few words it’s just “After each unit finishes execute a callback that runs its test using current real state data and log the results”.
I see this as a higher level of logging or just another way for reporting.

The implementation is bad but fits its purpose and should work for simple ruby programs, i haven’t tested more complex things like rack applications or anything that actually matters.

Anyway, it was a fun little experiment which i still can’t figure if is of any use what so ever, if someone is interested go to the github repo, look at the readme and give it a spin.
Have fun!