LiveCode LessonsHow To - Step-By-Step Guides To Tasks In LiveCode TSNetHow to download multiple files in the background with tsNet

How to download multiple files in the background with tsNet

Introduction

In order to provide a smooth user experience, it is often helpful if any files (images, videos or other data files) that need to be retrieved across the network can be downloaded by the application seamlessly in the background.

The following lesson shows how you can download a series of files in parallel and keep of track of which ones are yet to be retrieved.

Lay out the Stack

Create a new stack then drag a button and a field onto it.

Set the name of the button to "Download Files" and the name of the field to "Status".

Save this stack.

Create the Script

Edit the script of the button that you placed on the stack by right clicking on the button and selecting "Edit script".

Add the following code.

local sMaxRequests
local sActiveRequests
local sDownloadArray
local sRequestedArray
local sDataDir
local sDownloadErrors
-- The JSON data file must be in the following format:
--
-- [
--   {
--     "URL": "<url to download>",
--     "status": "",
--     "filename": "<filename to save download to>"
--   },
--   {
--     "URL": "<url to download>",
--     "status": "",
--     "filename": "<filename to save download to>"
--   }
-- ]
--
on mouseUp
   -- Make sure we are not already downloading files in the background
   if sActiveRequests is not 0 then
      answer "Downloads currently in progress."
   end if
   
   -- Set a limit for the number of requests we want to perform at a time
   put 3 into sMaxRequests
   
   -- This will be set to true if an error occurs while downloading any of the files
   put false into sDownloadErrors
   
   -- Reset the array of requested downloads
   put empty into sRequestedArray

   -- Specify the folder that downloaded files will be stored in
   put specialFolderPath("Desktop") into sDataDir
   
   -- Retrieve the JSON data file which contains the files to download and the current download status of each
   -- and convert it to an array called sDownloadArray
   put URL("file:" & sDataDir & "/URLs.json") into tJsonData
   put JSONToArray(tJsonData) into sDownloadArray
   put "Downloading..." into field "Status"
   
   -- Begin downloading
   send "downloadURLs" to me in 0 milliseconds
end mouseUp
on downloadURLs
   put the number of lines in the keys of sDownloadArray into tArrayCount
   
   -- If we are not already downloading the maximum number of simultaneous files, loop through
   -- sDownloadArray and initiate a request to an entry that is not yet "downloaded"
   if sActiveRequests < sMaxRequests then
      repeat with x = 1 to tArrayCount
         if sDownloadArray[x]["status"] is not "downloaded" and sRequestedArray[x] is not true then
            put tsNetGet(x, sDownloadArray[x]["URL"], tHeaders, "transferComplete") into tResult
            
            -- Increment the number of active requests and flag this item as having been requested
            if tResult is empty then
               add 1 to sActiveRequests
               put true into sRequestedArray[x]
            end if
         end if
         
         -- Stop downloading new files if we have reach the maximum number of simultaneous downloads
         if sActiveRequests = sMaxRequests then 
            exit repeat
         end if
      end repeat
   end if
   
   -- If we have looped through the array and no downloads are occurring, we have finished processing
   -- all the files to download, so execute the "downloadsCompleted" handler
   if sActiveRequests = 0 then
      downloadsCompleted
      exit downloadURLs
   end if
   
   -- Keep looping if we are not yet finished
   send "downloadURLs" to me in 50 milliseconds
end downloadURLs
on transferComplete pID, pResult, pBytes, pCurlCode
   -- Decrement the number of active requests as one has just finished
   subtract 1 from sActiveRequests
   
   -- Check the status codes and look for success
   if pCurlCode is 0 and pResult is 200 then
      -- If successful, retrieve the downloaded data
      put tsNetRetrData(pID, tError) into tData
      
      if tError is empty then
         -- Downloaded data has been retrieved successfully, save the data to the filesystem
         -- and update the status of the URL in sDownloadArray 
         put "downloaded" into sDownloadArray[pID]["status"]
         put tData into URL("file:" & sDataDir & slash & sDownloadArray[pID]["filename"])
         
         -- Save the array back to the filesystem so that if the application is closed prior
         -- to all files being downloaded, we can pick up where we left off
         put ArrayToJSON(sDownloadArray,,true) into URL("file:" & sDataDir & "/URLs.json")
      end if
   else
      -- A transfer must have failed, flag the failure and update the URL's status in sDownloadArray
      put true into sDownloadErrors
      put "failed" into sDownloadArray[pID]["status"]
      put ArrayToJSON(sDownloadArray,,true) into URL("file:" & sDataDir & "/URLs.json")
   end if
   
   -- Always call tsNetCloseConn to release any memory associated with the transfer
   tsNetCloseConn pID
end transferComplete
on downloadsCompleted
   -- Check for any download errors during processing, and notify the user accordingly
   if sDownloadErrors is true then
      put "All downloads attempted, but at least one was unsuccessful. Click the button to retry any failed downloads."  into field "Status"
   else
      put "All downloads completed successfully." into field "Status"
   end if
end downloadsCompleted

Create a file

The button script above accesses a text file to retrieve and maintain the list of URLs that must be downloaded.

Open up your favourite text editor program and copy the following text into your file.  Name this file "URLs.json" and save it to your desktop.

[
  {
    "URL": "https://www.livecode.com",
    "filename": "livecode.txt",
    "status": ""
  },
  {
    "URL": "https://www.google.com",
    "filename": "google.txt",
    "status": ""
  },
  {
    "URL": "https://www.microsoft.com",
    "filename": "microsoft.txt",
    "status": ""
  },
  {
    "URL": "https://www.apple.com",
    "filename": "apple.txt",
    "status": ""
  }
]

Test

Now you are ready to test your coding and see the results.

Switch to Run mode and click the button.

More information

This example uses a text file in JSON format to maintain the list of URLs to download and their current download status.  However, this information could be stored in any number of formats.  It could also be retrieved from a remote URL or stored in a database.

0 Comments

Add your comment

E-Mail me when someone replies to this comment