|
NewModules
Implement your own modules
Featured Table of contents
Implement your own modulesPlowshare is designed with modularity in mind, so it should be easy for other programmers to add new modules. Study the code of any of the existing modules (i.e. 2shared) and create your own. Some hosters are exporting a public API (formalized way for downloading or uploading), if it is available, it can save you lots of time calling this API, instead of simulating a web browser. For example: HotFile. Script templateEach module implements services for one sharing site:
The module must declare the following global variables: MODULE_XXX_REGEXP_URL Depending module features, some additional variables should also be declared: MODULE_XXX_DOWNLOAD_OPTIONS MODULE_XXX_DOWNLOAD_RESUME MODULE_XXX_DOWNLOAD_FINAL_LINK_NEEDS_COOKIE MODULE_XXX_UPLOAD_OPTIONS MODULE_XXX_UPLOAD_REMOTE_SUPPORT MODULE_XXX_DELETE_OPTIONS MODULE_XXX_LIST_OPTIONS Where XXX is the name of module (uppercase). No other global variable declaration is allowed. Module must export one to four entries point:
Downloading functionPrototype is: xxx_download() {
eval "$(process_options xxx "$MODULE_XXX_DOWNLOAD_OPTIONS" "$@")"
local COOKIEFILE=$1
local URL=$2
...
}Notes:
Arguments:
Warning: If function does not need a cookie file, do not delete cookie file provided as argument, plowdown will take care of this. When a link is correct, function should return 0 and echo one or two arguments, corresponding to file URL and filename: echo "$FILE_URL" echo "$FILENAME" $FILENAME can be empty, or even not echoed at all. If so, plowdown will guess filename from provided $FILE_URL. If cookie file is required for final download MODULE_XXX_DOWNLOAD_FINAL_LINK_NEEDS_COOKIE must be set to yes. File URL must return the final link (that's it, a link that return a 200 HTTP code, without redirection). Use curl -I and grep_http_header_location when necessary. Note: $FILE_URL will be encoded right after. So don't bother about weird characters. For example: spaces chars will be translated to %20 for you. Possible return valuesModule can return the following codes:
Additional error codes (returned by plowdown only, module download function should not return these):
Guidelines
Uploading functionPrototype is: xxx_upload() {
eval "$(process_options xxx "$MODULE_XXX_UPLOAD_OPTIONS" "$@")"
local COOKIEFILE=$1
local FILE=$2
local DESTFILE=$3
...
PAGE=$(curl_with_log ...) || return
...
}Notes:
Arguments:
Warning: If function does not need a cookie file, do not delete cookie file provided as argument, plowup will take care of this. When requested file has been successfully uploaded, function should return 0 and echo one or three lines. echo "$DL_URL" echo "$DEL_URL" echo "$ADMIN_URL_OR_CODE" $DEL_URL and $ADMIN_URL_OR_CODE are optional (can be empty or not echoed at all). Example1 (seen in depositfiles module): echo "$DL_LINK" echo "$DEL_LINK" Example2 (seen in 2shared module): echo "$FILE_URL" echo echo "$FILE_ADMIN" Possible return valuesModule can return the following codes:
Additional error codes (returned by plowup only, module upload function should not return these):
Guidelines
MAX_SIZE=... #Â hardcoded value or parse it in html page (if possible)
SIZE=$(get_filesize "$FILE")
if [ $SIZE -gt $MAX_SIZE ]; then
log_debug "file is bigger than $MAX_SIZE"
return $ERR_SIZE_LIMIT_EXCEEDED
fiDeleting functionPrototype is: xxx_delete() {
eval "$(process_options xxx "$MODULE_XXX_DELETE_OPTIONS" "$@")"
local COOKIEFILE=$1
local URL=$2
...
}Notes:
Argument:
Warning: If function does not need a cookie file, do not delete cookie file provided as argument, plowdel will take care of this. There is not output for this function. When file has been successfully deleted, function should return 0. Possible return valuesModule can return the following codes:
Additional error codes (returned by plowdel only, module delete function should not return these):
Guidelines
Listing functionPrototype is: xxx_list() {
eval "$(process_options xxx "$MODULE_XXX_LIST_OPTIONS" "$@")"
local URL=$1
local RECURSE=${2:-"0"}
...
}Notes:
Arguments:
As result, function must echo download links (one URL per line). Possible return valuesModule can return the following codes:
Additional error codes (returned by plowlist only, module list function should not return these):
Guidelines
test "$2" && log_debug "recursive flag is not supported" Output debug messages (stderr)Do not use echo which is reserved for function return value(s). Use log_debug() or log_error(). You can use -vN command line option switch to change debug verbosity. Note: An intermediate verbosity level exists: log_notice(), it is reserved to core functions, do not use it inside modules. Module argumentsIf MODULE_XXX_DOWNLOAD_OPTIONS / MODULE_XXX_UPLOAD_OPTIONS / MODULE_XXX_DELETE_OPTIONS or MODULE_XXX_LIST_OPTIONS is not empty, you must process arguments at the beginning of the function using: eval "$(process_options xxx "$MODULE_XXX_LIST_OPTIONS" "$@")" This will modify $@ and affect values (if user specified them) to module option. Example: Assuming module source contain: MODULE_XXX_DELETE_OPTIONS=" AUTH,a:,auth:,USER:PASSWORD,User account" Assuming user is invoking plowdel with an account: $ plowdel -a 'user:password' 'http://www.sharing-site.com/?delete=12D45G5' Module function xxx_delete() will be called with the following arguments: $1='-a' $2='user:password' $3='http://www.sharing-site.com/?delete=12D45G5' after process_options call: $1='http://www.sharing-site.com/?delete=12D45G5' AUTH='user:password' curl functionThis is probably the most important command in plowshare API set. This wrapper function is calling curl real binary (let's call it true-curl) Arguments:
Note: curl_with_log is calling curl but force verbose level to 3. This is a specific usage for module upload function (should be called one time only). It's a good habit to always append || return for error handling. Examples: PAGE1=$(curl "http://www.google.com") || return
# Get remote content and take cookies (if any)
PAGE2=$(curl -c "$COOKIE_FILE" "$URL") || return
# Get remote content, provides and append cookie entries
PAGE3=$(curl -c "$COOKIE_FILE" -b 'lang=en' "$URL") || return
PAGE4=$(curl -c "$COOKIE_FILE" -b "$COOKIE_FILE" "$URL") || return
PAGE5=$(curl "${URL}?param=1") || return
# or
PAGE5=$(curl --get --data 'param=1' "$URL") || returnNotes:
Temporary files are deleted in case of errorFirst example using -H/--dump-headers: HEADERS=$(create_tempfile) || return HTML=$(curl -H "$HEADERS" http://...) || return rm -f "$HEADERS" If something goes wrong in curl (network issue or anything else), $HEADERS will be deleted for you. Remember, it's only if an error occurs. On curl's success nothing is deleted (as expected). Another classic example if using -o/--output: CAPTCHA_URL='http://...' CAPTCHA_IMG=$(create_tempfile '.png') || return curl -o "$CAPTCHA_IMG" "$CAPTCHA_URL" || return ... rm -f "$CAPTCHA_IMG" If something append when retrieving captcha image, curl will delete temporary file for you. Split long data stringDATA="action=validate&uid=123456&recaptcha_challenge_field=$CHALLENGE&recaptcha_response_field=$WORD" RESULT=$(curl -b "$COOKIE_FILE" --data "$DATA" "$URL") || return Consider passing several -d/--data argument instead of one (order is not important). RESULT=$(curl -b "$COOKIE_FILE" -d 'action=validate' \
-d "uid=123456" \
-d "recaptcha_challenge_field=$CHALLENGE" \
-d "recaptcha_response_field=$WORD" \
"$URL") || returnIs better for maintenance. Auxiliar functionsYou can see a full list of plowshare public API on NewModules2 wiki page. core.sh script provides usual auxiliar functions.
Goal here, is not calling non portable commands in modules. Function: post_loginIt is a useful function for registered accounts because ID information is stored inside cookie. This function will send the HTML form for you, It takes 4 or 5 arguments. Arguments:
Example: # comes from command line AUTH="mylogin:mypassword" # important: notice simple quote, $USER and $PASSWORD must not be interpreted. LOGIN_DATA='login=1&redir=1&username=$USER&password=$PASSWORD' LOGIN_URL="https://xxx.com/login.php" # or simply use $(create_tempfile) COOKIES=/tmp/my_cookie_file post_login "$AUTH" "COOKIES" "$LOGIN_DATA" "$LOGIN_URL" >/dev/null Results:
A common usage is (snippet taken from filesonic module): LOGIN_RESULT=$(post_login "$AUTH" "$COOKIE_FILE" "$LOGIN_DATA" 'http:///www.fileserve.com/login.php') || return If no password is provided, post_login will prompt for one. Warning: Having $?=0 does not mean that your account is valid, it just means that the request (in a HTTP protocol point of view) have been successful. For detecting bad login/password, you'll have to parse returned HTML content or sometimes cookie file. Note: Sometimes, parsing LOGIN_RESULT can be useful to distinguish free account from premium account. Sometimes parsing cookie (looking for specific entry in it) can help too. Use case 1 (seen in netload.in module)An empty $LOGIN_RESULT is not necessarily an error. You can get for example a HTTP redirection. You could eventually follow this redirection by giving '-L' option to curl: LOGIN_RESULT=$(post_login "$AUTH" "$COOKIE_FILE" "$LOGIN_DATA" "$BASEURL/login.php" -L) || return Use case 2 (seen in mediafire module)You already have valid entries in $COOKIEFILE (language for example) and you want keeping them. LOGIN_RESULT=$(post_login "$AUTH_FREE" "$COOKIEFILE" "$LOGIN_DATA" \
"$BASE_URL/dynamic/login.php?popup=1" -b "$COOKIEFILE") || returnWithout this additional -b "$COOKIEFILE" given to curl, cookie file would be overwritten. Functions: match and matchiArguments:
Results:
'I' letter stand for case-insensitive match. match does not use sed command, so you don't have to escape "/" (slash) character. Regexp are basic posix. Reserved characters (to escape) are: . * [ ]. Coding convention is to use the shortest write: match 'foo' "$HTML_PAGE" && ... // right
$(match 'foo' "$HTML_PAGE") && ... // wrong (useless subshell creation)
match '\(foo\)' "$HTML_PAGE" && ... // wrong (useless parenthesis)
if (! match 'You are ' "$HTML"); then // wrong (useless subshell creation)
...
fiTypical use: if ! match '/js/myfiles\.php/' "$PAGE"; then
log_error "not a folder"
return $ERR_FATAL
fiSimple examples: match '[0-9][0-9]\+' 'Wait 19 seconds' // true match '[0-9][0-9]\+' 'Wait 9 seconds' // false match 'times\?' 'One time ago' // true match 's/n' 'yes/no' // true match '(euros)' '3.5 (euros)' // true match '\[euros\]' '3.5 [euros]' // true More examples (seen in modules): match '^http://download' "$LOCATION" // ^ matches begining of line match 'errno=999$' "$LOCATION" // $ matches end of line match '.*/#!index|' "$URL" // . means any character match 'File \(deleted\|not found\|ID invalid\)' "$ERROR" Functions: parse, parse_all, parse_quiet, parse_all_quiet and parse_lastThe first function will return first match, second one will return all matches (multiline result). sed command in internally used here. Arguments:
Results:
Regexp are basic posix. Reserved characters (to escape) are: . * / [ ]. Examples: ID=$(echo "$HTML_PAGE" | parse 'name="freeaccountid"' 'value="\([[:digit:]]*\)"') HOSTERS=$(echo "$FORM" | parse_all 'checked' '">\([^<]*\)<br') MSG=$(echo "$RESPONSE" | parse_quiet "ERROR:" "ERROR:[[:space:]]*\(.*\)") FIXME: Add examples with ^ and $ Should I use parse or parse_quiet? Use xxx_quiet functions when parsing failure is a normal behavior, for example, parsing an optional value. Typical use: OPT_RESULT=$(echo "$HTML_PAGE" | parse_quiet 'id="completed"' '">\([^<]*\)<\/font>') If you actually require a result, do not put it quiet (you'll get a sed error message if parse fails): Typical use: WAIT_TIME=$(echo "$HTML_PAGE" | parse '^[[:space:]]*count=' "count=\([[:digit:]]\+\);") || return Note: Don't use these functions for HTML parsing. Consider using parse_tag and parse_attr functions family (see below #Parsing_HTML_markers and #Parsing_HTML_attributes). Functions: parse_line_after and parse_line_after_allArguments:
Results:
This is useful when filter regexp and catch regexp are not on the same line. Like all parse* functions, sed command in internally used here. This will be very useful if you want to grep this: <div class="dl_filename"> FooBar.tar.bz2</div> We can get the right line with filtering with dl_filename and apply your filename regexp on the second line (the line after). This will give: echo "$PAGE" | parse_line_after 'dl_filename' '\([^<]*\)' Another example: function js_fff() {
R4z5sjkNo = "http://...";
DelayTime = 60;
...Get URL with: DL_LINK=$(echo "$PAGE" | parse_line_after 'js_fff' '"\([^"]\+\)";') || return Get counter value with (this is some kind of parse_line_after_after): COUNT=$(echo "$PAGE" | parse_line_after 'js_fff' '=[[:space:]]*\([[:digit:]]\+\)' 2) || return Function: basename_urlGet basename (hostname) of an URL. A=$(basename_url 'http://code.google.com/p/plowshare/wiki/NewModules' # result: http://code.google.com B=$(basename_url 'http://code.google.com/' # result: http://code.google.com C=$(basename_url 'abc' # result: abc Functions: grep_http_header_location, grep_http_header_location_quietArgument:
Result:
If you think you reached the final url (let's call it $FINAL_URL) for download, and when you curl it (with -I/--head option), you got some HTTP answer like this: HTTP/1.1 301 Moved Permanently Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Location: /download/123/5687/final_filename.xyz Content-type: text/html Content-Length: 0 Connection: close Date: Sun, 17 Jan 2010 14:34:47 GMT Server: Apache Use grep_http_header_location to deal with this redirection. Have a look at sendspace module: HOST=$(basename_url "$FINAL_URL")
PATH=$(curl -I "$FINAL_URL" | grep_http_header_location) || return
echo "${HOST}${PATH}"Another example with absolute uri (comes from euroshare.eu): HTTP/1.1 302 Found Date: Sat, 10 Mar 2012 11:14:31 GMT Server: Apache/2.2.16 (Debian) Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Expires: Thu, 19 Nov 1981 08:52:00 GMT Set-Cookie: sid=61bu6nt3kkh9nsk92mg7otg501; expires=Sun, 11-Mar-2012 11:14:31 GMT; path=/ Location: http://s1.euroshare.eu/download/3598184/aXa2YWy3ytUhu3uVUsAQEgUzUDUseje3/5344113/myfile.zip Access-Control-Allow-Origin: * Access-Control-Allow-Headers: x-requested-with Access-Control-Allow-Headers: x-file-name Access-Control-Allow-Headers: content-type Vary: Accept-Encoding Content-Type: text/html FILE_URL=$(curl -I "$FINAL_URL" | grep_http_header_location) || return echo "$FILE_URL" Note: Like other *_quiet functions, grep_http_header_location_quiet is silent and do always return 0. Use this only on dedicated case. For example: FILE_URL=$(echo "$HTML_PAGE" | grep_http_header_location_quiet) || return
if [ -z "$FILE_URL" ]; then
... # not premiumFunction: grep_http_header_content_dispositionArgument:
Result:
Sharing websites often return their files as an attachment. curl doesn't care about Content-Disposition:. So, it will not parse this HTTP header but keeps url as name reference (see -O option documentation). $ curl http://p123.share-site.com/download/dl.php?id=123456456 # saved filename will be: "dl.php?id=123456456" The reason for that, is that link can have multiple attachments. Note: This is a difference between curl and wget. Note: This is not true anymore. Since curl 7.20.0, -J/--remote-header-name option has been added (you must combine it with -O/--remote-name). Plowshare does not use this for now. Have a look at divshare module: FILE_NAME=$(curl -I "$FILE_URL" | grep_http_header_content_disposition) || return Before plowdown core script make the final HTTP GET request, module is doing a HTTP HEAD request in order to parse attachment header and get filename. $ curl -I http://p123.share-site.com/download/dl.php?id=123456456 HTTP/1.0 200 OK Date: Sun, 28 Feb 2010 11:41:50 GMT Server: Apache Last-Modified: Mon, 12 Oct 2009 10:04:20 GMT ETag: 9852859-16341905311255341860 Cache-Control: max-age=30 Content-Disposition: attachment; filename="kop_standard.pdf" Accept-Ranges: bytes Content-Length: 412848 Vary: User-Agent Keep-Alive: timeout=300, max=100 Connection: keep-alive Content-Type: application/octet-stream Notice that some sharing sites does not an allow HTTP HEAD requests. Restricting web server is maybe a security concern? There is a possible workaround: HTTP 1.1 protocol allow to make to HTTP GET request and specify a byte range. FILE_NAME=$(curl -i -r 0-99 "$FILE_URL" | grep_http_header_content_disposition) || return This is not very classy, but this can work, except if sharing site only allow one (and only one) HTTP request to that final URL (uploaded.to for example). In that case you couldn't get attachment filename. Function: grep_http_header_content_locationArgument:
Result:
Get HTTP "Content-Location:" value. Function: grep_http_header_content_typeArgument:
Result:
Get HTTP "Content-Type:" value. Functions: parse_cookie, parse_cookie_quietArguments:
Result:
This is often used to get account settings. Sometimes, for premium account, remote site adds an extra key in cookie file. So it can be convenient to differ free account from premium account. LOGIN_ID=$(parse_cookie 'Login' < "$COOKIEFILE") || return PASS_HASH=$(parse_cookie 'Password' < "$COOKIEFILE") || return # At this point You are sure that $LOGIN_ID and $PASS_HASH are valid (non empty) Note: Like other *_quiet functions, parse_cookie_quiet is silent and do always return 0. Use this only on dedicated case. For example: USERNAME=$(parse_cookie_quiet 'login' < "$COOKIEFILE")
if [ -z "$USERNAME" ]; then
... # invalid account
return $ERR_LOGIN_FAILED
fiParsing HTML markersArguments:
$1 can be omitted. Only tag name is passed single argument. Result:
The all functions are for multiline content, one tag is parsed per line. Important: If you have several matching tags on the same line, the first one is taken. Remember that this is line oriented, if beginning tag and ending are not on the same line, it won't work. It's not perfect, but for now, it covers all our need. Examples: LINE='... <a href="link1">Link number 1</a> <a href="javascript:;">Link number 2</a>' LINK1=$(echo "$LINE" | parse_tag a) || return # 1st link returned LINE='... <b></b> ...' CONTENT=$(echo "$LINE" | parse_tag b) || return # error, return called # Nested elements: take the deepest one! WAIT_MSG='<span id="foo">Wait <span id="bar">30</span> seconds</span>' WAIT_TIME=$(echo "$WAIT_MSG" | parse_tag span) || return # 30 Note: "parse_tag b" is equivalent to "parse_tag . b" and "parse_tag b b". Parsing HTML attributesArguments:
Result:
The all functions are for multiline content, one attribute is parsed per line. Important: If you have several matching attribute on the same line, the last one is taken. Examples: IMG='<img href="http://foo.com/bar.jpg" alt="">' CONTENT=$(echo "$IMG" | parse_attr img alt) || return # error, return called PAGE='<a href="http://...">click here to download</a>' LINK=$(echo "$PAGE" | parse_attr 'download' 'href') || return log_debug "[$LINK]" # [http://...] IMG='<img href="http://foo.com/bar.jpg" id = image_id>' ID=$(echo "$IMG" | parse_attr 'id') || return Note: "parse_attr b" is equivalent to "parse_attr . b" and "parse_attr b b". Some websites return page as a single big line of HTML (without any eol). As parse_xxx functions are per-line oriented, proper parsing can be difficult. Two functions exists: break_html_lines and break_html_lines_alt (more aggressive) to split single line HTML. Parsing HTML formscore.sh script provides some functions. Assume here, for our example, curl retrieved HTML page and stored it in $HTML_PAGE variable.
You are strongly encouraged to append regular || return error handling. Example: FORM_URL=$(grep_form_by_order "$HTML_PAGE" 1 | parse_form_action) || return # We are sure here, that $HTML_PAGE has a form with an action attribute # We can safely use $FORM_URL now Note: parse_form_input_by_id_quiet, parse_form_input_by_name_quiet and parse_form_input_by_type_quiet are available. Like other *_quiet functions, there's not error message and do always return 0. You generally use them when you want to parse a html form field with possible empty value. For example: FORM_SID=$(echo "$FORM_HTML" | parse_form_input_by_id_quiet 'sid') #Â $FORM_SID can be 0 for anonymous users and it can be defined (non empty) for account user. Captcha helper functionscore.sh script provides some functions. captcha_processArguments:
In most usual cases, $3 should be left empty, the best image viewer (using X or not) is chosen. Current solving methods:
Other methods are private. Results:
Typical usage: ($CAPTCHA_IMG is a valid image file) local WI WORD ID
WI=$(captcha_process "$CAPTCHA_IMG" ocr_digit) || return
{ read WORD; read ID; } <<<"$WI"
rm -f "$CAPTCHA_IMG"Note: If something goes wrong ($? is not 0), argument image file is deleted. recaptcha_processArgument:
Results:
Typical usage: local PUBKEY WCI CHALLENGE WORD ID
PUBKEY='6Lftl70SAAABAItWJueKIVvyG5QfLgmAgtKgVbDT'
WCI=$(recaptcha_process $PUBKEY) || return
{ read WORD; read CHALLENGE; read ID; } <<<"$WCI"positive or negative acknowledgeEach you call captcha_process or recaptcha_process, you get a transaction id as result. Once captcha result submitted, you must acknowledge captcha transaction. There are two functions: captcha_ack or captcha_nack. Argument:
Typical usage: if match ... wrong captcha ...; then
captcha_nack $ID
log_error "Wrong captcha"
return $ERR_CAPTCHA
fi
captcha_ack $ID
log_debug "correct captcha"JSON parsingOfficial format standard: RFC4627. If you know nothing about JavaScript Object Notation, try this: curl http://twitter.com/users/bob.json | python -mjson.tool Functions: parse_json, parse_json_quietSimple and limited JSON parsing. sed command in internally used here. Arguments:
Results:
Important notes:
Simple usage: FILE_URL=$(echo "$JSON" | parse_json 'downloadUrl') || return Function match_json_trueArguments:
Results:
This will literally match for true boolean token, "true" string token or any number will be considered as false. # Assuming that a curl request can result one of two $JSON content:
# JSON='{"err":"Entered digits are incorrect."}'
# JSON='{"ok":true,"dllink":"http:\/\/www.share-me.com\/..."}'
if ! match_json_true 'ok' "$JSON"; then
ERR=$(echo "$JSON" | parse_json_quiet err)
test "$ERR" && log_error "Remote error: $ERR"
return $ERR_FATAL
fi
log_debug "ok answer..."Module command-line switchesThere are some specific modules options (see MODULE_XXX_DOWNLOAD_OPTIONS / MODULE_XXX_UPLOAD_OPTIONS / MODULE_XXX_DELETE_OPTIONS or MODULE_XXX_LIST_OPTIONS): AuthenticationAUTH,a:,auth:,USER:PASSWORD,Premium account AUTH_FREE,b:,auth-free:,USER:PASSWORD,Free account Most of the time, when a module can deal with both free and premium, we will see a single option: AUTH,a:,auth:,USER:PASSWORD,User account For delete, it's quite usual that authentication is mandatory for deleting files, you'll see: AUTH,a:,auth:,USER:PASSWORD,User account (mandatory) Download usual optionsLINK_PASSWORD,p:,link-password:,PASSWORD,Used in password-protected files Ask for password if not supplied: log_debug "File is password protected"
if [ -z "$LINK_PASSWORD" ]; then
LINK_PASSWORD="$(prompt_for_password)" || return
fiUpload usual optionsLINK_PASSWORD,p:,link-password:,PASSWORD,Protect a link with a password DESCRIPTION,d:,description:,DESCRIPTION,Set file description FROMEMAIL,,email-from:,EMAIL,<From> field for notification email TOEMAIL,,email-to:,EMAIL,<To> field for notification email Guidelines
Coding rulesBash pitfall: quote variable when it contains several linesWAIT_TIME=$(echo $WAIT_HTML | parse 'foo' '.. \(...\) ..') Won't give you expected answer if $WAIT_HTML is multiline (which is most of the time the case). You should write instead: WAIT_TIME=$(echo "$WAIT_HTML" | parse 'foo' '.. \(...\) ..') Consider this example for understanding: $ MYS=$(seq 3) $ echo "$MYS" 1 2 3 $ echo $MYS 1 2 3 $ echo $MYS | xxd 0000000: 3120 3220 330a 1 2 3. Bash pitfall: no local keyword with || returnUnfortunately, this is not correct: local HTML_PAGE=$(curl "$URL") || return If curl function returns an error, it won't be catched by || return because of the local keyword. local HTML_PAGE ... HTML_PAGE=$(curl "$URL") || return is correct. Bash pitfall: avoid single statement "&& ||"Â test$ set -- test $ [ -z "$1" ] && echo empty || echo nonempty nonempty $ set -- $ [ -z "$1" ] && echo empty || echo nonempty empty $ set -- test $ [ -z "$1" ] || echo nonempty && echo empty nonempty empty $ set -- $ [ -z "$1" ] || echo nonempty && echo empty empty Looks like "&& ||" is better than "|| &&". But imagine that echo empty does not return $?=0: $ set -- $ [ -z "$1" ] && echo empty; false || echo nonempty empty nonempty Finally, classic if/then/else/fi is not so bad! if [ -z "$1"]; then
echo empty
else
echo nonempty
fiPortabilityPlowshare is running on lots of unix/linux systems. There is always several ways to write bash code. We try to keep compatibility with busybox shell. Things to take care or avoid in your module functions:
Bash specific construct to avoid:
Busybox specific pitfalls:
Try being compliant with bash 3.x. Interesting reading: Miscellaneous remarks
Why is this so complicated and constrained?It's because we want to be portable as much as possible. We loose flexibility, but it can be run on slow and old embedded hardware, this is the original starting point of the project. But maybe plowshare with bash 4.0 as minimum requirement will pop-up one day... Coding styleGeneral Rules
Naming Rules
Strongly recommended guidelines1. if/then construct and while/do are on the same line. 2. Restrict usage of curly braces: test "$FILE_URL" || { log_error "location not found"; return $ERR_FATAL; }should be written: if test "$FILE_URL"; then
log_error "location not found"
return $ERR_FATAL
fi3. In comment, insert a space character of # symbol #get id of file (wrong) # Get id of file (right) 4. Proper indentation on continued lines HTML=$(curl -b "$COOKIE_FILE" 'http://www.foo.bar/long...url...') \ || return $ERR_FATAL should be written: HTML=$(curl -b "$COOKIE_FILE" 'http://www.foo.bar/long...url...') || \
return $ERR_FATAL5. Simple quote strings as much as possible If there is no variable referencing of course! local BASE_URL="http://shareme.com" (wrong) local BASE_URL='http://shareme.com' (right) TestingTest and retest your module. Little check-list of possible cases:
Other concerns:
If possible, update test suite. One of these:
Send your contributionCreate a patch file (git diff or diff -u, ...) and create a new issue. Commit message should have the following standard format: module_name: one-line summary If possible, be more precise in the first line et add matter in parentheses. For example: mediafire (download): add password link support rapidshare (upload): fix account support External documentation
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
► Sign in to add a comment
Thank You very mush =) !! Jonathan.
sweet! if i knw how to do one thing, its bash :D..