Issue 8: Tests must be completelly redone.
Status:  Fixed
Owner: ----
Closed:  Aug 2008
Reported by edmundo...@gmail.com, Feb 10, 2008
What steps will reproduce the problem?
1. Go to the application directory.
2. Type rake test:plugins PLUGIN=substruct

What is the expected output? What do you see instead?
It should execute the tests, but it just abort.

What version of the product are you using? On what operating system?
substruct_rel_0-97 on Ubuntu Feisty Fawn (7.04)

Please provide any additional information below.

Output on console:
./vendor/plugins/substruct/test/unit/../test_helper.rb:4: undefined method
`config' for Engines:Module (NoMethodError)
...
rake aborted!
Command failed with status (1): [/usr/bin/ruby1.8 -Ilib:test "/usr/lib/ruby...]

I don't have sure when the tests stopped to be maintained or how it was
different in the past, but the documentation of engines plugin shows that
its only needed two lines of code to make it work. It's pretty the same
thing but using a temporary path defined by a method inside
Engines::Testing. So we get rid of config options but continue to use
fixtures inside the plugin directory (why do they copy the fixtures to /tmp?)

The tests doesn't run, it references things that doesn't exists anymore,
the fixtures appears to have been erased. So appears the it should be
redone completelly.

I think this is a very important thing, as now theres no references to see
if a patch breaks something or not, just trying to make the unit tests for
three classes I already found some issues, and the fixes are beeing tested
by hand. 

Feb 10, 2008
Project Member #1 subim...@gmail.com
Well, the tests were never really that robust ;)

If someone wants to work up a testing framework I'll commit it ASAP.
Status: Accepted
Labels: -Priority-Medium Priority-Low
Feb 11, 2008
#2 edmundo...@gmail.com
I'm doing it, but it will take a while, I will need to look at all the code to see
how it should work and make all the unit, functional and integration tests and at the
end, mocks to the billing and shipping part.
It probably will break several times before its done and if commited earlier will be
incomplete (but I think that an incomplete usefull thing is a good thing).

The good thing is that it don't mess with the code :)
Apr 8, 2008
#3 edmundo...@gmail.com
By now its just a sample, the file has the name "tests_YYMMDD.tar.gz", has the test
and tasks directories (I changed the test helper and added a task to measure coverage
using rcov).

Its not representable using diff, so it includes the whole files.

Current status:
262 tests, 1013 assertions, 0 failures, 0 errors
+----------------------------------------------------+-------+-------+--------+
|                  File                              | Lines |  LOC  |  COV   |
+----------------------------------------------------+-------+-------+--------+
|app/controllers/application.rb                      |     5 |     5 | 100.0% |
|app/controllers/store_controller.rb                 |    11 |     9 |  22.2% |
|app/helpers/application_helper.rb                   |    38 |    31 |   9.7% |
|app/models/image.rb                                 |    16 |    13 |  46.2% |
|app/models/order.rb                                 |    75 |    52 |   5.8% |
|...substruct/app/controllers/accounts_controller.rb |    37 |    30 |  26.7% |
|...bstruct/app/controllers/admin/base_controller.rb |     9 |     7 | 100.0% |
|...pp/controllers/admin/content_nodes_controller.rb |   176 |   130 | 100.0% |
|...struct/app/controllers/admin/files_controller.rb |    65 |    47 | 100.0% |
|...ruct/app/controllers/admin/general_controller.rb |     4 |     4 | 100.0% |
|...truct/app/controllers/admin/orders_controller.rb |   325 |   208 |  26.4% |
|.../app/controllers/admin/preferences_controller.rb |   129 |    76 | 100.0% |
|...uct/app/controllers/admin/products_controller.rb |   247 |   169 | 100.0% |
|...t/app/controllers/admin/promotions_controller.rb |    62 |    51 | 100.0% |
|...ct/app/controllers/admin/questions_controller.rb |    55 |    48 | 100.0% |
|...struct/app/controllers/admin/roles_controller.rb |    47 |    36 | 100.0% |
|...uct/app/controllers/admin/sections_controller.rb |    79 |    63 | 100.0% |
|...bstruct/app/controllers/admin/tags_controller.rb |    79 |    63 | 100.0% |
|...struct/app/controllers/admin/users_controller.rb |   118 |    78 | 100.0% |
|...ruct/app/controllers/content_nodes_controller.rb |    73 |    55 |  18.2% |
|...ubstruct/app/controllers/countries_controller.rb |    14 |    12 | 100.0% |
|...ubstruct/app/controllers/customers_controller.rb |   212 |   147 |  12.9% |
|...ins/substruct/app/controllers/main_controller.rb |     7 |     5 | 100.0% |
|...s/substruct/app/controllers/paypal_controller.rb |    33 |    27 |  11.1% |
|...ubstruct/app/controllers/questions_controller.rb |    29 |    22 |  27.3% |
|...ns/substruct/app/controllers/store_controller.rb |   484 |   324 |  13.6% |
|.../controllers/substruct_application_controller.rb |    59 |    43 |  41.9% |
|...ugins/substruct/app/helpers/admin/base_helper.rb |    65 |    51 |  21.6% |
|...gins/substruct/app/helpers/application_helper.rb |    66 |    52 |  32.7% |
|...or/plugins/substruct/app/helpers/order_helper.rb |   160 |   115 |   7.8% |
|vendor/plugins/substruct/app/models/asset.rb        |    13 |     7 | 100.0% |
|vendor/plugins/substruct/app/models/blog.rb         |    14 |     8 | 100.0% |
|vendor/plugins/substruct/app/models/cart.rb         |    88 |    63 | 100.0% |
|vendor/plugins/substruct/app/models/content_node.rb |    56 |    30 | 100.0% |
|vendor/plugins/substruct/app/models/country.rb      |    14 |    11 | 100.0% |
|vendor/plugins/substruct/app/models/image.rb        |    19 |    12 | 100.0% |
|vendor/plugins/substruct/app/models/item.rb         |    25 |     9 | 100.0% |
|vendor/plugins/substruct/app/models/order.rb        |   718 |   494 |  11.7% |
|...or/plugins/substruct/app/models/order_account.rb |   140 |    91 |  78.0% |
|...or/plugins/substruct/app/models/order_address.rb |    44 |    33 | 100.0% |
|.../plugins/substruct/app/models/order_line_item.rb |    43 |    33 | 100.0% |
|...gins/substruct/app/models/order_shipping_type.rb |    48 |    35 | 100.0% |
|...ns/substruct/app/models/order_shipping_weight.rb |    13 |     5 | 100.0% |
|vendor/plugins/substruct/app/models/order_user.rb   |   151 |   102 |  94.1% |
|...or/plugins/substruct/app/models/orders_mailer.rb |    32 |    28 |  42.9% |
|vendor/plugins/substruct/app/models/page.rb         |     5 |     2 | 100.0% |
|vendor/plugins/substruct/app/models/preference.rb   |    54 |    34 | 100.0% |
|vendor/plugins/substruct/app/models/product.rb      |   178 |   118 | 100.0% |
|...or/plugins/substruct/app/models/product_image.rb |     7 |     4 | 100.0% |
|vendor/plugins/substruct/app/models/promotion.rb    |    83 |    52 | 100.0% |
|vendor/plugins/substruct/app/models/question.rb     |     4 |     3 | 100.0% |
|vendor/plugins/substruct/app/models/right.rb        |    12 |     3 | 100.0% |
|vendor/plugins/substruct/app/models/role.rb         |    19 |    14 | 100.0% |
|vendor/plugins/substruct/app/models/section.rb      |    37 |    24 | 100.0% |
|vendor/plugins/substruct/app/models/snippet.rb      |     6 |     2 | 100.0% |
|vendor/plugins/substruct/app/models/tag.rb          |    61 |    39 | 100.0% |
|vendor/plugins/substruct/app/models/user.rb         |    94 |    56 |  96.4% |
|vendor/plugins/substruct/app/models/user_upload.rb  |    57 |    28 | 100.0% |
|vendor/plugins/substruct/app/models/variation.rb    |    49 |    24 | 100.0% |
|...or/plugins/substruct/app/models/wishlist_item.rb |     7 |     4 | 100.0% |
|vendor/plugins/substruct/lib/pagination.rb          |   404 |   203 |  51.7% |
|vendor/plugins/substruct/lib/substruct.rb           |   225 |   109 |  24.8% |
|.../plugins/substruct/lib/substruct/login_system.rb |   111 |    50 |  56.0% |
|...or/plugins/substruct/lib/substruct_deprecated.rb |    31 |    24 |  16.7% |
+----------------------------------------------------+-------+-------+--------+
|Total                                               |  5641 |  3727 |  55.3% |
+----------------------------------------------------+-------+-------+--------+
55.3%   64 file(s)   5641 Lines   3727 LOC

tests_080408.tar.gz
176 KB   Download
Apr 8, 2008
Project Member #4 subim...@gmail.com
Added tests with revision #62

A handful of the tests are throwing errors or failing. I'm suspecting it's because your local copy has more of the 
patches applied than I currently to with the trunk.

Once I work through applying all the patches I'll come back to this and check for failing / errors / etc.
Status: Started
Apr 9, 2008
#5 edmundo...@gmail.com
See what you will commit, then I can sync back to trunk and see what is lacking.
Apr 9, 2008
Project Member #6 subim...@gmail.com
I'm handling some of these issues and cleaning up tests where I can, as I go.

Submitted some fixes in revision 75
Apr 9, 2008
#7 edmundo...@gmail.com
Ok, updated my working copy to r76, theres still five broken tests (they are harmless
as any test) but expect that for every not applied or even applied in a modified way,
exclusion of dead code (yes, I touch even dead code to fill coverage), etc, you break
at least one test.

When getting rid of something you can run rake substruct:test:coverage to see if some
part of the code is not being touched by tests anymore.
Apr 9, 2008
Project Member #8 subim...@gmail.com
Definitely - I think from here on out all new releases should be covered and tested properly. Mind patching up 
the remaining tests? Or would you rather me look @ them?


Apr 9, 2008
#9 edmundo...@gmail.com
Sure I can fix it, all of it are easy to fix but I will need to take a look around first:

1) Failure: test_should_not_remove_yourself(Admin::UsersControllerTest)
* I didn't saw how different you fixed this yet.

2) Failure: test_should_discover_if_content_node_is_a_blog_post(ContentNodeTest)
* Remove of dead code.

3) Failure: test_should_not_create_invalid_shipping_type(OrderShippingTypeTest)
4) Failure: test_should_not_create_invalid_shipping_type(OrderShippingWeightTest)
* These I will need to see if I will change the patch, the test, or both. These are
issues 64 and 65.

5) Error: test_should_not_create_invalid_user(UserTest):
* I didn't saw how different you fixed this yet too.

May 28, 2008
#10 edmundo...@gmail.com
Things are very quiet here, these past weeks. But for anyone who may interested, the
tests continues being done.

...
311 tests, 1216 assertions, 0 failures, 0 errors
+----------------------------------------------------+-------+-------+--------+
|                  File                              | Lines |  LOC  |  COV   |
+----------------------------------------------------+-------+-------+--------+
...
+----------------------------------------------------+-------+-------+--------+
|Total                                               |  5537 |  3659 |  70.9% |
+----------------------------------------------------+-------+-------+--------+
70.9%   60 file(s)   5537 Lines   3659 LOC


Jun 15, 2008
#12 edmundo...@gmail.com
Tests reached almost 90%, here goes a patch to them. I tested with my working copy
synchronized with r104, its expected yet that it gives 6 errors.

One error is about weight calculation that was not applied yet ( issue 69 ).

And the other 5 errors are about problems discussed in  issue 97  (errors that happens
when some methods are executed more than one time but without executing the
controller action that trigger it, it breaks mostly promotions processing). And about
ocasions where assigning an id to a relation directly don't refresh the related
object (doing a_thing.another_thing_id = 1 instead of a_thing.another_thing =
another_thing), so something that is expected to have changed wasn't refreshed it.

I have patches to all of these, but as I didn't finished testing the store_controller
yet, I cannot explain how the actions obfuscate these problems (I could not reproduce
these problems using the public interface), I didn't filled them yet.

I don't know it there will be problems applying this as a patch or I will need to
make a tar file again.

Anyway, executing this:
$ rake test:plugins PLUGIN=substruct

Should end up returning this:
...
356 tests, 1573 assertions, 6 failures, 0 errors
rake aborted!
...

Regards.
080615.patch
172 KB   View   Download
Aug 11, 2008
Project Member #13 subim...@gmail.com
Thanks again for all of the hard work on this Edmundo. Just rolled this into trunk
with r108
Aug 12, 2008
#14 edmundo...@gmail.com
I will sync and take a look at the tests, mine can be almost be considered 100%, so
these are outdated (its a two month old patch). :)
Aug 14, 2008
#15 edmundo...@gmail.com
Here comes a new patch and its status, its synced with r116:

rake substruct:test:coverage
...
Finished in 87.927567 seconds.

  1) Failure:
test_should_buy_something(BuyerTest)
    [./vendor/plugins/substruct/test/integration/buyer_test.rb:205:in
`test_should_buy_something'
     /usr/lib/ruby/gems/1.8/gems/mocha-0.5.6/lib/mocha/test_case_adapter.rb:19:in
`__send__'
     /usr/lib/ruby/gems/1.8/gems/mocha-0.5.6/lib/mocha/test_case_adapter.rb:19:in `run'
    
/home/edmundo/workspace_aptana/trunk/vendor/rails/actionpack/lib/action_controller/integration.rb:547:in
`run']:
UNEXPECTED SECOND ORDER ITEM QUANTITY AFTER CHECKOUT.
<1> expected but was
<2>.

  2) Failure:
test_should_return_total_weight(OrderTest)
    [./vendor/plugins/substruct/test/unit/order_test.rb:667:in
`test_should_return_total_weight'
     /usr/lib/ruby/gems/1.8/gems/mocha-0.5.6/lib/mocha/test_case_adapter.rb:19:in
`__send__'
     /usr/lib/ruby/gems/1.8/gems/mocha-0.5.6/lib/mocha/test_case_adapter.rb:19:in `run'
    
/home/edmundo/workspace_aptana/trunk/vendor/rails/activesupport/lib/active_support/testing/default.rb:7:in
`run']:
<0.9> expected but was
<44.45>.

  3) Failure:
test_should_set_promo_code(OrderTest)
    [./vendor/plugins/substruct/test/unit/order_test.rb:168:in
`test_should_set_promo_code'
     /usr/lib/ruby/gems/1.8/gems/mocha-0.5.6/lib/mocha/test_case_adapter.rb:19:in
`__send__'
     /usr/lib/ruby/gems/1.8/gems/mocha-0.5.6/lib/mocha/test_case_adapter.rb:19:in `run'
    
/home/edmundo/workspace_aptana/trunk/vendor/rails/activesupport/lib/active_support/testing/default.rb:7:in
`run']:
Fixed rebate with minimum cart value verification error.
<696.6625> expected but was
<726.75>.

381 tests, 1967 assertions, 3 failures, 0 errors
+----------------------------------------------------+-------+-------+--------+
|                  File                              | Lines |  LOC  |  COV   |
+----------------------------------------------------+-------+-------+--------+
|app/controllers/application.rb                      |     6 |     6 | 100.0% |
|...substruct/app/controllers/accounts_controller.rb |    38 |    31 |  74.2% |
|...bstruct/app/controllers/admin/base_controller.rb |    11 |     8 | 100.0% |
|...pp/controllers/admin/content_nodes_controller.rb |   176 |   130 | 100.0% |
|...struct/app/controllers/admin/files_controller.rb |    65 |    47 | 100.0% |
|...ruct/app/controllers/admin/general_controller.rb |     4 |     4 | 100.0% |
|...truct/app/controllers/admin/orders_controller.rb |   325 |   208 |  98.6% |
|.../app/controllers/admin/preferences_controller.rb |   120 |    72 | 100.0% |
|...uct/app/controllers/admin/products_controller.rb |   246 |   169 | 100.0% |
|...t/app/controllers/admin/promotions_controller.rb |    62 |    51 | 100.0% |
|...ct/app/controllers/admin/questions_controller.rb |    55 |    48 | 100.0% |
|...struct/app/controllers/admin/roles_controller.rb |    47 |    36 | 100.0% |
|...uct/app/controllers/admin/sections_controller.rb |    79 |    63 | 100.0% |
|...bstruct/app/controllers/admin/tags_controller.rb |    79 |    63 | 100.0% |
|...struct/app/controllers/admin/users_controller.rb |   109 |    78 | 100.0% |
|...ruct/app/controllers/content_nodes_controller.rb |    73 |    55 |  98.2% |
|...ubstruct/app/controllers/countries_controller.rb |    14 |    12 | 100.0% |
|...ubstruct/app/controllers/customers_controller.rb |   248 |   176 |  84.7% |
|...ins/substruct/app/controllers/main_controller.rb |     7 |     5 | 100.0% |
|...lugins/substruct/app/controllers/order_helper.rb |   160 |   115 | 100.0% |
|...s/substruct/app/controllers/paypal_controller.rb |    33 |    27 | 100.0% |
|...ubstruct/app/controllers/questions_controller.rb |    32 |    25 | 100.0% |
|...ns/substruct/app/controllers/store_controller.rb |   498 |   334 |  96.1% |
|.../controllers/substruct_application_controller.rb |    66 |    47 |  68.1% |
|...ugins/substruct/app/helpers/admin/base_helper.rb |    65 |    51 |  21.6% |
|...gins/substruct/app/helpers/application_helper.rb |    93 |    70 |  98.6% |
|vendor/plugins/substruct/app/models/asset.rb        |    13 |     7 | 100.0% |
|vendor/plugins/substruct/app/models/blog.rb         |    14 |     8 | 100.0% |
|vendor/plugins/substruct/app/models/cart.rb         |    88 |    63 | 100.0% |
|vendor/plugins/substruct/app/models/content_node.rb |    56 |    30 | 100.0% |
|vendor/plugins/substruct/app/models/country.rb      |    14 |    11 | 100.0% |
|vendor/plugins/substruct/app/models/image.rb        |    19 |    12 | 100.0% |
|vendor/plugins/substruct/app/models/item.rb         |    36 |    13 | 100.0% |
|vendor/plugins/substruct/app/models/order.rb        |   720 |   497 |  97.8% |
|...or/plugins/substruct/app/models/order_account.rb |   146 |    96 | 100.0% |
|...or/plugins/substruct/app/models/order_address.rb |    44 |    35 | 100.0% |
|.../plugins/substruct/app/models/order_line_item.rb |    43 |    33 | 100.0% |
|...gins/substruct/app/models/order_shipping_type.rb |    70 |    43 | 100.0% |
|...ns/substruct/app/models/order_shipping_weight.rb |    11 |     5 | 100.0% |
|...lugins/substruct/app/models/order_status_code.rb |    13 |    11 | 100.0% |
|vendor/plugins/substruct/app/models/order_user.rb   |   151 |   102 | 100.0% |
|...or/plugins/substruct/app/models/orders_mailer.rb |    33 |    29 | 100.0% |
|vendor/plugins/substruct/app/models/page.rb         |     5 |     2 | 100.0% |
|vendor/plugins/substruct/app/models/preference.rb   |    55 |    35 |  88.6% |
|vendor/plugins/substruct/app/models/product.rb      |   166 |   112 | 100.0% |
|...or/plugins/substruct/app/models/product_image.rb |     7 |     4 | 100.0% |
|vendor/plugins/substruct/app/models/promotion.rb    |    83 |    52 | 100.0% |
|vendor/plugins/substruct/app/models/question.rb     |     6 |     5 | 100.0% |
|vendor/plugins/substruct/app/models/right.rb        |    12 |     3 | 100.0% |
|vendor/plugins/substruct/app/models/role.rb         |    18 |    13 | 100.0% |
|vendor/plugins/substruct/app/models/section.rb      |    37 |    24 | 100.0% |
|vendor/plugins/substruct/app/models/snippet.rb      |     6 |     2 | 100.0% |
|vendor/plugins/substruct/app/models/tag.rb          |    61 |    39 | 100.0% |
|vendor/plugins/substruct/app/models/user.rb         |    87 |    55 |  96.4% |
|vendor/plugins/substruct/app/models/user_upload.rb  |    57 |    28 | 100.0% |
|vendor/plugins/substruct/app/models/variation.rb    |    38 |    20 | 100.0% |
|...or/plugins/substruct/app/models/wishlist_item.rb |     7 |     4 | 100.0% |
|vendor/plugins/substruct/lib/pagination.rb          |   404 |   203 |  51.7% |
|vendor/plugins/substruct/lib/smtp_tls.rb            |    65 |    56 |  14.3% |
|vendor/plugins/substruct/lib/substruct.rb           |   225 |   109 |  78.0% |
|.../plugins/substruct/lib/substruct/login_system.rb |   111 |    50 |  64.0% |
|vendor/plugins/substruct/lib/substruct/version.rb   |     9 |     8 | 100.0% |
|...or/plugins/substruct/lib/substruct_deprecated.rb |    30 |    24 |  33.3% |
+----------------------------------------------------+-------+-------+--------+
|Total                                               |  5671 |  3774 |  91.3% |
+----------------------------------------------------+-------+-------+--------+
91.3%   63 file(s)   5671 Lines   3774 LOC


Some untouched parts are libs and helpers, theres dead code, and some few new added
methods (maybe 2) that don't have tests.

About the errors:

Error 1 is about  issue 111 . It shows its broken.
Error 2 is about  issue 69 , Its about weight calculation.
Error 3 is about  issue 103 . It wasn't properly fixed.


Thats it.
080814.patch
39.4 KB   View   Download
Aug 15, 2008
Project Member #16 subim...@gmail.com
In trunk now. Failing with one test which seems to be  issue 103 .
Aug 15, 2008
Project Member #17 subim...@gmail.com
Tests pass 100%. Closing. Thanks for all the great work Edmundo!
Status: Fixed