<cribeiro> Hi Remi. Saw your invitation, visiting to say hello :-)
<Remi> Hi Carlos
<Remi> I'm just working on the new cherrypy.org site :-)
<Remi> If the DNS change has propagated, you should be able to see it
<Remi> I'm just using the Trac site as the main CherryPy site.
<Remi> This way it will be easier for other people to contribute and make changes to the website
<cribeiro> Ok. That's great news.
<cribeiro> There is something that I would like to talk about...
<Remi> sure
<cribeiro> I'm facing some problems trying to figure out how to fit my own routines into CP2, specially due to the way the testing is currently structured.
<Remi> which routines for instance ?
<cribeiro> Let me re-state the problem :-)
<cribeiro> I may be wrong. Really. But I think that the current test set for CP2 is more like an "integration test", or "system test", than a real unit test in the sense of test-driven-development.
<Remi> You're talking about unit tests for individual functions in the source code ?
<cribeiro> I never did TDD before, and I'm having a hard time trying to figure out how to adapt my own coding style to it.
<cribeiro> Yes, individual unit tests, maybe not for individual functions, but for modules, or groups of functions.
<Remi> I'm not against them, but I think that nothing replaces high-level tests (actually writing CherryPy apps, launching the server, requesting pages and checking the result)
<cribeiro> (About TDD: having read about it, I think that it may be the definitive cure for the endless "run test/fix one bug/ do it again" development style).
<Remi> But that doesn't mean we can't have module-level unittests either
<cribeiro> Humm. I think that we have a (healthy) disagreement here :-) Something good for a debate, sure. I think both are useful. The high-level integration test doesn't substitute unit-tests.
<cribeiro> Oh-oh. As I was typing my msg, I didn't see your comment. So we don't really disagree :-)
<cribeiro> (offtopic: it must be at least three or four years since my last IRC chat. Seems weird after such a long time)
<Remi> :-)
<Remi> Well, I think that high-level tests are in a sense more complete than unit-tests because they really test real-life scenarios ...
<Remi> Have you looked at some tools for doing module-level tests ?
<cribeiro> The problem is that they can't fully exercise each unit. By their very nature, it would mean a combinatorial explosion. Think about it: let us say that there are 10 modules, each one with 10 tests. The total of module level tests is a simple sum, right? one hundred. But the 
  combinations are exponential.
<cribeiro> So what happens is that the integration test is (by its very definition) simplistic. It can't exercise all posible variations. A good combination of module tests and a well-written system level test gives a much better coverage.
<Remi> That's true.
<cribeiro> The basic tools are the same; basically unittest.py. The "problem" with module level tests is building the test bed -- I mean, the environment to exercise the module.
<Remi> I see ...
<Remi> Well, each module could have a "test" function that would test the functions defined in this module.
<Remi> Then "unittest.py" would just call the "test" function for each module.
<cribeiro> But this also helps to highlight one of the advantages that test-drive-developmento evangelists keep talking about (...suspense pause...)
<cribeiro> By writing the test bed *outside* of the unit 
  itself, one must think really hard on the modularization. One example. My test for the URL mapping routine required me to check all necessary parameters. I had to avoid the temptation to use the CPG structure. The resulting code is much more modular than it would be if it was tested inside the current CP2 environment.
<Remi> I see ...
<Remi> So are you saying that we need to change the architecture of CherryPy ?
<cribeiro> No. The current architecture is fine.
<cribeiro> And I *DON'T* intend to touch it, or to propose anything so far fetched :-)
<cribeiro> The "right way" to make unit tests -- and I'm *far* from being a expert in the subject -- seems to be write a separate module, for each of the modules being tested. Or at least that's wht Mark Pilgrim proposs in his Dive Into Python book (available online).
<cribeiro> In this sense, what I propose is really simple. For each source file in the CP2 structure, write an equivalent 
  test file, that exercises all relevant routines into that source file. 
<Remi> Well, that's kind of what the "high-level" tests do then ... 
<Remi> If we wanted, we could write a "high-level" test that tests each function
<cribeiro> (btw - just checked the site. looks interesting. but at some point, and I may be dreaming, it would be nice to design the main page with CP2, don't you think? But it may be a distraction at this point...)
<Remi> Although in some cases, starting a whole server and requesting a page would probably be overkill just to test a single function...
<cribeiro> Yes (it's overkill to start the server just to exercise a function)
<Remi> But nothing prevents us from just importing a module and calling one of its function instead of starting the server ...
<Remi> About the new site: I thought of having the main page 
  served by a CP2 site and the other ones as Wiki (so people can contribute) but then I thought it wasn't much of a point to have a CP2 site with just one page :-) So I decided it wasn't worth it in the end and I went with Trac all the way ...
<cribeiro> It must be *clear* that I'm not talking about substituting the tests that are inplace now. They are important. But there are lots of good reasons to write more comprehensive tests, at least for some parts of CP2.
<Remi> But don't worry, we'll have real-life CP2 sites soon ... (Porting Python-Hosting.com to CP2 is next on my list :-)
<cribeiro> Back to the testing. My proposals: (1) rename the current unittest to something like integrationtest (if you don't mind, of course); (2) write a parallel unittest set, with placeholder files for each file in the CP2 directory.
<cribeiro> I'm also thinking about taking a look at other (bigger) Python projects to check how they do it.
<cribeiro> (another off topic remark: do you save these IRC sessions? It would be interesting to have them saved into 
  the main site, at some point -- a link to the "IRC archives". Better if done automatically, of course... Am I dreaming?)
<Remi> Well, I wasn't going to save them but why not ...
<cribeiro> Hummm.. it's about time for me to leave, now. Family affairs :-) Now that I finally installed mIRC, I'll be back for a chat, surely.
<Remi> It would be easy to just copy&paste this into the Wiki ...
<cribeiro> A few times, yes. In the long run, it's easy to forget. But it's something intersting to have... even if the chat goes off topic, unless something private is mentioned (which it shouldn't), but even so, a quick select-del fixes it.
<Remi> ok. I'll put it in the wiki. Bye