Took a long, hard look at the code, and sadly it seems very susceptible to 
different text encodings breaking things. I'm rather upset with myself on this 
one; I should have been thinking ahead. I haven't heard back from the user who 
originally reported the issue, but I am now strongly convinced this was her 
issue.

Whether or not this is the problem, I am making changes to the code that will 
force it to be more internally consistent, using UTF-8 everywhere 
text-to-binary and binary-to-text conversions are made. The import/export 
format *should* already be safe, but internally many things relied on the 
system default. According to this page:

http://developer.android.com/reference/java/lang/String.html#getBytes%28%29

"Converts this string to a byte array using the default encoding as specified 
by the file.encoding system property. If the system property is not defined, 
the default encoding is ISO8859_1 (ISO-Latin-1). If 8859-1 is not available, an 
ASCII encoding is used."

These encodings are unacceptable for international use. Unfortunately, I 
strongly suspect that this change will break many international users' data. 
Even worse, the Android Market provides no useful feedback on where downloads 
are coming from, so I have no idea how many users this may affect.

Update from same user on September 3rd: "I was using the cyanogenmod flavour of 
Android 2.1 on my last phone. I do not know if there was a specific locale on 
that phone. I am using Samsung's stock Android 2.1 now." She also said she 
would run a number of tests concerning my suspicions and would get back to me, 
but I haven't heard anything from her since.

I think I'm going to go on record that we have no plans on officially 
supporting custom ROMs (i.e. stuff like cyanogen). It's bad enough having to 
support all the crazy configurations from all the official sources; I think if 
people are flashing devices to do non-standard things, they should be on their 
own. ;D

I am moving forward with the plan to move all text encoding to UTF-8. I assumed 
a system as "modern" as Android would default to UTF-8, but as noted above, 
that apparently isn't the case. As noted above, moving from one encoding to 
another will definitely break things (except for devices where the encoding 
defaults to something like ISO-Latin-1 or ASCII, where the characters should be 
roughly the same as UTF-8). The hard part is coding things to prevent this 
breakage from happening, maintaining backward compatibility, while moving 
things forward to prevent this from happening again.

The current plan is to implement an "upgrade manager" that will check a new 
SharedPreferences file to the version number that Cryptnos previous ran under. 
Obviously, the first time Cryptnos runs with this new code in place, there will 
be no preferences file, so we'll have a special case. Successive runs will be 
able to selectively "upgrade" the data when it discovers that it is running a 
newer version than the last time it ran. After the upgrade, the upgrade manager 
will store the new (current) version number in the preferences so the upgrade 
check will be short-circuited the next time it is run.

In the case when a brand new installation is encountered (i.e. no prior version 
number and no data in the database), the upgrade manager will automatically 
force the user to use UTF-8 for text encoding, regardless of the system 
default. Thus, all future users going forward should be safe.

In the case where no prior version is found but data is, we'll need to notify 
the user of the situation: We want to "upgrade" you to UTF-8, but that may or 
may not break your current passwords. Most(?) users should be unaffected. In 
this case, we need to make the "upgrade" optional and let the user decide. They 
should be strongly encouraged to regenerate the passwords and use UTF-8 going 
forward, but we need to understand that this could present serious problems for 
people with many passwords stored in the database.

Anyone wishing to help debug this issue, please perform the following steps:

1. Download the following APK: 
http://code.google.com/p/cryptnos-for-android/downloads/detail?name=CharsetTest.
apk&can=2&q=
2. Install the APK on your device. Note that you will need to enable 
"sideloading" of apps (or installing applications from unknown sources, i.e. 
not the Market) in your device settings: Settings -> Applications -> Unknown 
Sources
3. Run the app (CharsetTest). It will produce a simple screen listing the 
system default text encoding and all text encoding character sets your device 
supports. (It should be a *very* long list.)
4. AT A MINIMUM, send me the default encoding listed at the top of the list. 
You can post it here, shoot me an e-mail, or use any of the other usual 
channels. If you get any form of error message, please report that as well.
5. You can also tap the Export to SD Card button at the top of the screen. This 
will export the same data to a text file on the root of your SD card 
(charset_test.log). You can copy that file from your device and attach it here, 
e-mail to me, etc. Obviously, this is preferred as we'll get more information, 
but the system default is what we really need to know.

This test app is small and harmless. If anyone cares I can release its source, 
but I didn't want to add it to the SVN repo since it's a quick-and-dirty test 
program. The permissions dialog states that it wants to "read phone state and 
identity"; I have no idea what that's showing up, especially since it doesn't 
use any such information and doesn't request any permissions except access to 
the SD card. It might be because we're reading the system "file.encoding" 
property, but I figured that would be benign enough not to warrant such a 
warning. Anyway, it should be safe to ignore the "phone calls" permission; it's 
not listening on your conversations, I promise. Feel free to delete this app 
once you've reported the data.

Thanks in advance to anyone who helps out.

Alternate testing option in lieu of the previous comment (for those who don't 
want to install code from "unofficial" sources):

1. Go to the Android Market and search for "Dalvik Explorer". Download & 
install this app.
2. Open Dalvik Explorer. Tap Charsets.
3. The top entry in the list is the default character set (suffixed by 
"(default)"). Please note and report this value.

I've checked the default character sets on several real devices and emulators, 
and every single one of them reported UTF-8. No reports from users in the field 
yet, however. That said, the user that reported the initial problem did mention 
she moved from a non-standard mod ("cyanogenmod") to a standard Android 
install. It is possible the modded device used a different default character 
set than UTF-8. This *could* explain why there haven't been any more reports of 
this problem, that most people trying Cryptnos are using standard vanilla 
Android. Either that, or they haven't been upgrading devices. Too early to tell 
yet.

Any user reports that can help this will be greatly appreciated.

This issue was updated by revision r11.

Added new system preferences file to allow the user to change the text 
character encoding used by Cryptnos. The AdvancedSettingsActivity provides the 
interface to the user. Changed the use of text encodings throughout the 
application to use this preference in most places. Added the UpgradeManager 
class primarily to provide a sane way to add this to existing installations, 
but also provide a way to add and remove setting information when the user 
upgrades the application.

This issue was updated by revision r12.


More tweaks for text encoding issues. Forced UTF-8 for import/export XML 
format; should have been doing this all along (XML header even declares), but 
for some reason I was using the default encoding. Changed the encoding choices 
a couple other places too; storage (DB, import/export) should always use UTF-8 
but password generations should use the user's preference. Added a few UI 
tweaks here and there, like prompt text, spelling corrections, and adding a 
warning dialog for the Advanced Settings activity.

Tentatively set this to version 1.2.0b (beta), but haven't tagged this as 1.2.0 
in SVN yet.

OK, I've uploaded a new APK with a tentative version number of 1.2.0b (for 
BETA). This is *NOT* an official 1.2.0 release. However, I think it fixes the 
whole "wrong password" issue... sort of. The new rules are that for storage 
(database, import/export) Cryptnos will *ALWAYS* use UTF-8 for text encoding. 
For password generation, however, it will use the user's preference, which 
defaults (if no preference has been set or the application has never been used 
before) to UTF-8 as well. If there's no preference and the system default is 
something other than UTF-8, the user is prompted for what to do. Note, however, 
that every test I've been able to perform has stated that UTF-8 is the de facto 
standard encoding for "vanilla" Android installs, so I suspect this will only 
affect users using "mods" as mentioned above. Since I really haven't gotten any 
feedback from any users, including the original reporter of the error, this is 
the best I can do.

I'll wait to see if there's any feedback from anyone. If I don't hear anything 
for a "reasonable" amount of time, I'll assume no one's found any problems and 
I'll bump the official release to 1.2.0 with this code. There are a few other 
updates I'm considering making, but fixing this problem is a much higher 
priority than creeping features.

http://code.google.com/p/cryptnos-for-android/downloads/detail?name=Cryptnos_1.2
.0b.apk&can=2&q=

Another new beta is now up. This doesn't change anything with respect to this 
issue, but the new file manager selection code should be working where it 
wasn't before. If there aren't any more comments, I'll likely release this to 
the Market within a week or two.

http://code.google.com/p/cryptnos-for-android/downloads/detail?name=Cryptnos_1.2
.0b2.apk&can=2&q=

OK, the official version 1.2.0 has been released and pushed out to the main 
site and the Android Market. There's not much different between it and beta 2, 
but if you downloaded the betas you should probably switch to the official 
release just in case. I'm going to leave this issue open just a little longer 
to make sure nobody reports that the issue is continuing, but I plan to close 
it soon.

OK, it's been several weeks now. I'm going ahead and closing this issue. Please 
post any additional problems with this issue here and I'll reopen it if 
necessary. Thanks again to those who provided testing and information.