Download in Bulk

To unlock local access to the International Future Universe dataset, open the Pricing page of your organization and subscribe to the following data packages:

• International Future Universe History by QuantConnect
• US Futures Security Master by QuantConnect

You need billing permissions to change the organization's subscriptions.

After you subscribe to local access, to download the International Future Universe data, follow these steps:

1. Log in to the Algorithm Lab.
2. On the CLI tab of the dataset listing, use the CLI Command Generator to generate your download command and then copy it.
3. Open a terminal in your organization workspace and then run the command from the CLI Command Generator.

To download the US Futures Security Master, run:

$ lean data download --dataset "US Futures Security Master"

After you bulk download the International Future Universe dataset, new daily updates are available at 7 AM Eastern Time (ET) after each trading day. To unlock local access to the data updates, open the Pricing page of your organization and subscribe to the International Future Universe Updates by QuantConnect data package. You need billing permissions to change the organization's subscriptions.

After you subscribe to dataset updates, to update your local copy of the International Future Universe dataset, use the CLI Command Generator to generate your download command and then run it in a terminal in your organization workspace. Alternatively, instead of directly calling the lean data download command, you can place the following Python script in the data directory of your organization workspace and run it to update your data files.

import os
from datetime import datetime, time, timedelta
from pytz import timezone
from os.path import abspath, dirname
os.chdir(dirname(abspath(__file__)))

OVERWRITE = False

def __get_start_date() -> str:
    dir_name = f"future/hkfe/universes"
    files = [] if not os.path.exists(dir_name) else sorted(os.listdir(dir_name)) 
    return files[-1].split(".")[0] if files else '19980101'

def __get_end_date() -> str:
    now = datetime.now(timezone("US/Eastern"))
    if now.time() > time(7, 0):
        return (now - timedelta(1)).strftime("%Y%m%d")
    print('New data is available at 07:00 AM EST')
    return (now - timedelta(2)).strftime("%Y%m%d")

if __name__ == "__main__":
    start, end = __get_start_date(), __get_end_date()
    if start >= end:
        exit("Your data is already up to date.")
            
    command = f'lean data download --dataset "International Future Universe" --data-type "Bulk" --start {start} --end {end}'
    if OVERWRITE:
        command += " --overwrite"
    print(command)
    os.system(command)

The preceding script checks the date of the most recent International Future Universe data you have. If there is new data available, it downloads the new data files.

To update your local copy of the US Futures Security Master, run:

$ lean data download --dataset "US Futures Security Master"

The International Future Universe dataset is about 1 GB in size. We structure the data files so there is one file per Future per day. For an example, see the Data / future / hkfe / universes / hsi / 20131028.csv file in the LEAN repository.

To view the prices of the datasets, see the Pricing tab of the International Future Universe and US Futures Security Master dataset listings.