pistachiomedia-simpledb-export - Project Hosting on Google Code

Older Aug 27, 2008 issue 4 (--nexttoken / -n as option) commented on by builtbydave - Great idea. Great idea. Aug 27, 2008 issue 5 (--limit / -l as option) Status changed by builtbydave - Status: Verified Status: Verified Aug 27, 2008 issue 1 (ItemName (primary key) isn't exported) Status changed by builtbydave - Ok. Let's not create an option unless requested by users. Status: Verified Ok. Let's not create an option unless requested by users. Status: Verified Aug 27, 2008 issue 8 (-u / -p terminology) Status changed by builtbydave - Sorry, thought they were related. In future will add as new issue. Status: Verified Sorry, thought they were related. In future will add as new issue. Status: Verified Aug 27, 2008 issue 8 (-u / -p terminology) Status changed by stroppytux - Fixed. In future could these please be new bugs as they dont directly relate the the original bug. Status: Fixed Fixed. In future could these please be new bugs as they dont directly relate the the original bug. Status: Fixed Aug 27, 2008 r13 (Fixed bug #8. Removed the interactive option that was clashi...) committed by stroppytux - Fixed bug #8 . Removed the interactive option that was clashing with the new id option Fixed bug #8 . Removed the interactive option that was clashing with the new id option Aug 27, 2008 issue 8 (-u / -p terminology) Status changed by builtbydave - Since what looks like r10 I'm getting this error: Traceback (most recent call last): File "sdbexp.py", line 183, in <module> parser.add_option("-i", "--interactive", action="store_true", dest="is_interactive", default=False, help="run the script interactively") File "/usr/lib/python2.5/optparse.py", line 1014, in add_option self._check_conflict(option) File "/usr/lib/python2.5/optparse.py", line 989, in _check_conflict option) optparse.OptionConflictError: option -i/--interactive: conflicting option string(s): -i Status: Started Since what looks like r10 I'm getting this error: Traceback (most recent call last): File "sdbexp.py", line 183, in <module> parser.add_option("-i", "--interactive", action="store_true", dest="is_interactive", default=False, help="run the script interactively") File "/usr/lib/python2.5/optparse.py", line 1014, in add_option self._check_conflict(option) File "/usr/lib/python2.5/optparse.py", line 989, in _check_conflict option) optparse.OptionConflictError: option -i/--interactive: conflicting option string(s): -i Status: Started Aug 26, 2008 issue 9 (queryWithAttributes Implementation) reported by stroppytux - We need to migrate the code to using the new queryWithAttributes option. http://aws.typepad.com/aws/2008/08/amazon-simpledb.html. This will reduce the complexity of the attribute retrieval. We need to migrate the code to using the new queryWithAttributes option. http://aws.typepad.com/aws/2008/08/amazon-simpledb.html. This will reduce the complexity of the attribute retrieval. Aug 26, 2008 r12 (Changed --max longopt to --maxnumberofitems.) committed by stroppytux - Changed --max longopt to --maxnumberofitems. Changed --max longopt to --maxnumberofitems. Aug 26, 2008 issue 5 (--limit / -l as option) Status changed by stroppytux - Changed in rev #12. The longopt for this is now set to maxnumberofitems. Status: Fixed Changed in rev #12. The longopt for this is now set to maxnumberofitems. Status: Fixed Aug 26, 2008 issue 4 (--nexttoken / -n as option) Status changed by stroppytux - I have renamed the options value to -n and nexttoken, and redone the option description. I think it would be better if the option was checked to see if it was a filename or a standard string. Then users can enter a filename or a token value. I will implement this in the next major revision. Status: Accepted I have renamed the options value to -n and nexttoken, and redone the option description. I think it would be better if the option was checked to see if it was a filename or a standard string. Then users can enter a filename or a token value. I will implement this in the next major revision. Status: Accepted Aug 26, 2008 r11 (Fixed issue #4. The --start -s option has been renamed to -n...) committed by stroppytux - Fixed issue #4. The --start -s option has been renamed to -n --nexttoken and the help description has been modified to show it requires a file not the token value. Fixed issue #4. The --start -s option has been renamed to -n --nexttoken and the help description has been modified to show it requires a file not the token value. Aug 26, 2008 issue 8 (-u / -p terminology) Status changed by stroppytux - Fixed in rev 10. The username and password are now set using -i --id and -k --key. Note: the code still uses options.username and options.password Status: Fixed Fixed in rev 10. The username and password are now set using -i --id and -k --key. Note: the code still uses options.username and options.password Status: Fixed Aug 26, 2008 r10 (Fixed issue #8. The username and key are now set using -i --...) committed by stroppytux - Fixed issue #8 . The username and key are now set using -i --id and -k --key. Fixed issue #8 . The username and key are now set using -i --id and -k --key. Aug 26, 2008 issue 1 (ItemName (primary key) isn't exported) Status changed by stroppytux - Renamed the primary key to ItemName. I think creating an option to toggle the key will just be confusing to users. Status: Fixed Renamed the primary key to ItemName. I think creating an option to toggle the key will just be confusing to users. Status: Fixed Aug 26, 2008 r9 (Fixed issue #1. The primary key is now called ItemName) committed by stroppytux - Fixed issue #1 . The primary key is now called ItemName Fixed issue #1 . The primary key is now called ItemName Aug 25, 2008 issue 4 (--nexttoken / -n as option) Status changed by builtbydave - We should fall in-line with Amazon's terminology to avoid confusion. -s TOKEN, --start=TOKEN should become: -nt TOKEN, --nexttoken=TOKEN If the --help description "set the start token" read "set the start token file" hopefully we can avoid any confusion with people entering "-nt=rO0ABXNyACdjb20uYW1hem9uLnNkcy5RdWV..." Status: Started We should fall in-line with Amazon's terminology to avoid confusion. -s TOKEN, --start=TOKEN should become: -nt TOKEN, --nexttoken=TOKEN If the --help description "set the start token" read "set the start token file" hopefully we can avoid any confusion with people entering "-nt=rO0ABXNyACdjb20uYW1hem9uLnNkcy5RdWV..." Status: Started Aug 25, 2008 issue 8 (-u / -p terminology) reported by builtbydave - We should fall in-line with Amazon's terminology to avoid confusion. -u USER, --user=USER set the AWS_ACCESS_KEY_ID variable -p PASS, --pass=PASS set the AWS_SECRET_ACCESS_KEY variable should become: -id set AWS_ACCESS_KEY_ID -key set AWS_SECRET_ACCESS_KEY We should fall in-line with Amazon's terminology to avoid confusion. -u USER, --user=USER set the AWS_ACCESS_KEY_ID variable -p PASS, --pass=PASS set the AWS_SECRET_ACCESS_KEY variable should become: -id set AWS_ACCESS_KEY_ID -key set AWS_SECRET_ACCESS_KEY Aug 25, 2008 issue 5 (--limit / -l as option) Status changed by builtbydave - We should fall in-line with Amazon's terminology to avoid confusion. The 'verbose option' should be "MaxNumberOfItems". Status: Started We should fall in-line with Amazon's terminology to avoid confusion. The 'verbose option' should be "MaxNumberOfItems". Status: Started Aug 25, 2008 issue 1 (ItemName (primary key) isn't exported) Status changed by builtbydave - We should fall in-line with Amazon's terminology to avoid confusion. By default we should output this column as "ItemName", no "id". In a later version maybe we build this in as an option for the user to decide? Status: Started We should fall in-line with Amazon's terminology to avoid confusion. By default we should output this column as "ItemName", no "id". In a later version maybe we build this in as an option for the user to decide? Status: Started Aug 25, 2008 issue 6 (Multiple threads) Status changed by builtbydave - Status: Verified Status: Verified Aug 25, 2008 issue 3 (Default output to stdout (& stderr?)) Status changed by builtbydave - Status: Verified Status: Verified Aug 25, 2008 issue 2 (--domain / -d as option) Status changed by builtbydave - Status: Verified Status: Verified Aug 23, 2008 issue 6 (Multiple threads) commented on by builtbydave - Cool, I'm happy with that :) Cool, I'm happy with that :) Aug 23, 2008 issue 5 (--limit / -l as option) commented on by builtbydave - Happy with -m, however looking through the Amazon docs they use "MaxNumberOfItems", for the 'verbose option' should be use that to fall in-line with them? Happy with -m, however looking through the Amazon docs they use "MaxNumberOfItems", for the 'verbose option' should be use that to fall in-line with them? Aug 23, 2008 issue 1 (ItemName (primary key) isn't exported) commented on by builtbydave - Nice. I'm wondering if contrary to what I said earlier we should call this "ItemName", in-line with what SimpleDB calls it just to avoid adding extra terms. Thoughts? Nice. I'm wondering if contrary to what I said earlier we should call this "ItemName", in-line with what SimpleDB calls it just to avoid adding extra terms. Thoughts? Aug 23, 2008 issue 6 (Multiple threads) Status changed by stroppytux - Added a hard limit cap of 80. This could be tinkered with. Status: Fixed Added a hard limit cap of 80. This could be tinkered with. Status: Fixed Aug 23, 2008 r8 (Updated comments and set a hard limit of 80 on the thread co...) committed by stroppytux - Updated comments and set a hard limit of 80 on the thread count. Updated comments and set a hard limit of 80 on the thread count. Aug 23, 2008 issue 7 (Export token on part download) reported by stroppytux - We need to export the token file when we set the maximum quantity of items to download like we do when we receive a KeyboardInterrupt exception. We need to export the token file when we set the maximum quantity of items to download like we do when we receive a KeyboardInterrupt exception. Aug 23, 2008 issue 6 (Multiple threads) changed by stroppytux - Due to the way boto is built, we need to set a limit on the quantity of threads people can use. If, for instance, you set a thread count of 100, an exception is raised due to boto trying to open to many files at once. A thread cap should be set to something like 80, but we need to find the correct limit. (I know 50 doesn't raise the exception). Status: Accepted Labels: Type-Defect Priority-Critical −Type-Enhancement −Priority-Medium Due to the way boto is built, we need to set a limit on the quantity of threads people can use. If, for instance, you set a thread count of 100, an exception is raised due to boto trying to open to many files at once. A thread cap should be set to something like 80, but we need to find the correct limit. (I know 50 doesn't raise the exception). Status: Accepted Labels: Type-Defect Priority-Critical −Type-Enhancement −Priority-Medium Aug 23, 2008 issue 5 (--limit / -l as option) Status changed by stroppytux - Added properly in #7. We can now set the max downloads using -m or --max command line options Status: Fixed Added properly in #7. We can now set the max downloads using -m or --max command line options Status: Fixed Aug 23, 2008 issue 4 (--nexttoken / -n as option) Status changed by stroppytux - Added in #7 Status: Fixed Added in #7 Status: Fixed Aug 23, 2008 r7 (Implimented a starting token to continue downloads and set a...) committed by stroppytux - Implimented a starting token to continue downloads and set a limit on the quantity of tokens to download Implimented a starting token to continue downloads and set a limit on the quantity of tokens to download Aug 23, 2008 issue 4 (--nexttoken / -n as option) commented on by stroppytux - Due to the length of the tokens, I am going to save the token to a file called last_token.txt. I will then let people load the file if they need to. Due to the length of the tokens, I am going to save the token to a file called last_token.txt. I will then let people load the file if they need to. Aug 23, 2008 issue 5 (--limit / -l as option) commented on by stroppytux - Have decided to go for -m --max to set the maximum rows per query. If we need to change it later, its not a problem. Have decided to go for -m --max to set the maximum rows per query. If we need to change it later, its not a problem. Aug 23, 2008 issue 5 (--limit / -l as option) Status changed by stroppytux - Will have to have a look into the naming of this. We currently use -l to display a list of domains and -d to process a specific domain. Status: Started Will have to have a look into the naming of this. We currently use -l to display a list of domains and -d to process a specific domain. Status: Started Aug 23, 2008 issue 4 (--nexttoken / -n as option) Status changed by stroppytux - Looking into this. Will look for a way to catch an exception and return the last token processed. This might raise issues due to the threading nature, so will have to implement some kind of mutex on the token value. Status: Started Looking into this. Will look for a way to catch an exception and return the last token processed. This might raise issues due to the threading nature, so will have to implement some kind of mutex on the token value. Status: Started Aug 23, 2008 issue 1 (ItemName (primary key) isn't exported) Status changed by stroppytux - Added an ID field to the exported dictionary that stores the "name" value. If this needs to be renamed, let me know. Status: Fixed Added an ID field to the exported dictionary that stores the "name" value. If this needs to be renamed, let me know. Status: Fixed Aug 23, 2008 issue 1 (ItemName (primary key) isn't exported) Status changed by stroppytux - Will look into this Status: Accepted Will look into this Status: Accepted Aug 23, 2008 issue 6 (Multiple threads) Status changed by stroppytux - Fixed in revision 6 Status: Fixed Fixed in revision 6 Status: Fixed Aug 23, 2008 r6 (Fixed #6 and #1. Also refactored the way the system works to...) committed by stroppytux - Fixed #6 and #1. Also refactored the way the system works to enable exceptions to work properly and fixed to work on OSX. Fixed #6 and #1. Also refactored the way the system works to enable exceptions to work properly and fixed to work on OSX. Aug 23, 2008 r5 (More modifications to the threading system and some work on ...) committed by stroppytux - More modifications to the threading system and some work on getting the starting token working More modifications to the threading system and some work on getting the starting token working Aug 22, 2008 r4 (Started implimenting threaded connection using local connect...) committed by stroppytux - Started implimenting threaded connection using local connection class. Started implimenting threaded connection using local connection class. Aug 20, 2008 issue 6 (Multiple threads) reported by builtbydave - Jeff Barr from amazon says: > One suggestion is to run multiple threads in parallel. You should be able to make several > hundred requests per second without any problems at all. Should be able to start a number of threads to export quicker. Number of threads should be CLI option, with a sensible default set. Jeff Barr from amazon says: > One suggestion is to run multiple threads in parallel. You should be able to make several > hundred requests per second without any problems at all. Should be able to start a number of threads to export quicker. Number of threads should be CLI option, with a sensible default set. Aug 20, 2008 2 issues changed by stroppytux - Issues 2, 3 Issues 2, 3 Aug 20, 2008 r3 (Fixes issues 2 and three) committed by stroppytux - Fixes issues 2 and three Fixes issues 2 and three Aug 19, 2008 r2 (initial commit) committed by builtbydave - initial commit initial commit Aug 14, 2008 issue 5 (--limit / -l as option) Summary changed by builtbydave - Summary: --limit / -l as option Summary: --limit / -l as option Aug 14, 2008 issue 5 (-limit / -l as option) reported by builtbydave - Allow the currently hard coded 100 limit to set via an option. 100 should be the default. Allow the currently hard coded 100 limit to set via an option. 100 should be the default. Aug 14, 2008 issue 4 (--nexttoken / -n as option) Summary changed by builtbydave - Summary: --nexttoken / -n as option Summary: --nexttoken / -n as option

issue 8 (-u / -p terminology) Status changed by builtbydave - Since what looks like r10 I'm getting this error: Traceback (most recent call last): File "sdbexp.py", line 183, in <module> parser.add_option("-i", "--interactive", action="store_true", dest="is_interactive", default=False, help="run the script interactively") File "/usr/lib/python2.5/optparse.py", line 1014, in add_option self._check_conflict(option) File "/usr/lib/python2.5/optparse.py", line 989, in _check_conflict option) optparse.OptionConflictError: option -i/--interactive: conflicting option string(s): -i

Status: Started

Since what looks like r10 I'm getting this error: Traceback (most recent call last): File "sdbexp.py", line 183, in <module> parser.add_option("-i", "--interactive", action="store_true", dest="is_interactive", default=False, help="run the script interactively") File "/usr/lib/python2.5/optparse.py", line 1014, in add_option self._check_conflict(option) File "/usr/lib/python2.5/optparse.py", line 989, in _check_conflict option) optparse.OptionConflictError: option -i/--interactive: conflicting option string(s): -i

Status: Started

Older