Lean  $LEAN_TAG$
QuantConnect.Util.LeanData Class Reference

Provides methods for generating lean data file content More...

Static Public Member Functions

static string GenerateLine (IBaseData data, Resolution resolution, DateTimeZone exchangeTimeZone, DateTimeZone dataTimeZone)
 Converts the specified base data instance into a lean data file csv line. This method takes into account the fake that base data instances typically are time stamped in the exchange time zone, but need to be written to disk in the data time zone. More...
 
static DateTime ParseTime (string line, DateTime date, Resolution resolution)
 Helper method that will parse a given data line in search of an associated date time More...
 
static string GenerateLine (IBaseData data, SecurityType securityType, Resolution resolution)
 Converts the specified base data instance into a lean data file csv line More...
 
static Type GetDataType (Resolution resolution, TickType tickType)
 Gets the data type required for the specified combination of resolution and tick type More...
 
static bool IsCommonLeanDataType (Type baseDataType)
 Determines if the Type is a 'common' type used throughout lean This method is helpful in creating SubscriptionDataConfig More...
 
static bool IsValidConfiguration (SecurityType securityType, Resolution resolution, TickType tickType)
 Helper method to determine if a configuration set is valid More...
 
static string GenerateZipFilePath (string dataDirectory, Symbol symbol, DateTime date, Resolution resolution, TickType tickType)
 Generates the full zip file path rooted in the dataDirectory More...
 
static string GenerateZipFilePath (string dataDirectory, string symbol, SecurityType securityType, string market, DateTime date, Resolution resolution)
 Generates the full zip file path rooted in the dataDirectory More...
 
static string GenerateRelativeZipFileDirectory (Symbol symbol, Resolution resolution)
 Generates the relative zip directory for the specified symbol/resolution More...
 
static string GenerateRelativeFactorFilePath (Symbol symbol)
 Generates relative factor file paths for equities More...
 
static string GenerateRelativeZipFilePath (Symbol symbol, DateTime date, Resolution resolution, TickType tickType)
 Generates the relative zip file path rooted in the /Data directory More...
 
static string GenerateRelativeZipFilePath (string symbol, SecurityType securityType, string market, DateTime date, Resolution resolution)
 Generates the relative zip file path rooted in the /Data directory More...
 
static string GenerateRelativeUniversesDirectory (Symbol symbol)
 Generates the relative directory to the universe files for the specified symbol More...
 
static string GenerateUniversesDirectory (string dataDirectory, Symbol symbol)
 Generates the directory to the universe files for the specified symbol More...
 
static string GenerateZipEntryName (Symbol symbol, DateTime date, Resolution resolution, TickType tickType)
 Generate's the zip entry name to hold the specified data. More...
 
static string GenerateZipFileName (Symbol symbol, DateTime date, Resolution resolution, TickType tickType)
 Generates the zip file name for the specified date of data. More...
 
static string GenerateZipFileName (string symbol, SecurityType securityType, DateTime date, Resolution resolution, TickType? tickType=null)
 Creates the zip file name for a QC zip data file More...
 
static TickType GetCommonTickType (SecurityType securityType)
 Gets the tick type most commonly associated with the specified security type More...
 
static Symbol ReadSymbolFromZipEntry (Symbol symbol, Resolution resolution, string zipEntryName)
 Creates a symbol from the specified zip entry name More...
 
static TickType GetCommonTickTypeForCommonDataTypes (Type type, SecurityType securityType)
 Get the TickType for common Lean data types. If not a Lean common data type, return a TickType of Trade. More...
 
static SecurityType ParseDataSecurityType (string securityType)
 Matches a data path security type with the SecurityType More...
 
static bool TryParseSecurityType (string fileName, out SecurityType securityType, out string market)
 Parses file name into a Security and DateTime More...
 
static bool TryParsePath (string filePath, out Symbol symbol, out DateTime date, out Resolution resolution, out TickType tickType, out Type dataType)
 Parses file name into a Security and DateTime More...
 
static bool TryParsePath (string fileName, out Symbol symbol, out DateTime date, out Resolution resolution)
 Parses file name into a Security and DateTime More...
 
static IEnumerable< TradeBarAggregateTradeBars (IEnumerable< TradeBar > bars, Symbol symbol, TimeSpan resolution)
 Aggregates a list of second/minute bars at the requested resolution More...
 
static IEnumerable< QuoteBarAggregateQuoteBars (IEnumerable< QuoteBar > bars, Symbol symbol, TimeSpan resolution)
 Aggregates a list of second/minute bars at the requested resolution More...
 
static IEnumerable< QuoteBarAggregateTicks (IEnumerable< Tick > ticks, Symbol symbol, TimeSpan resolution)
 Aggregates a list of ticks at the requested resolution More...
 
static IEnumerable< TradeBarAggregateTicksToTradeBars (IEnumerable< Tick > ticks, Symbol symbol, TimeSpan resolution)
 Aggregates a list of ticks at the requested resolution More...
 
static CalendarInfo GetDailyCalendar (DateTime exchangeTimeZoneDate, SecurityExchange exchange, bool extendedMarketHours)
 Helper method to return the start time and period of a bar the given point time should be part of More...
 
static CalendarInfo GetDailyCalendar (DateTime exchangeTimeZoneDate, SecurityExchangeHours exchangeHours, bool extendedMarketHours)
 Helper method to return the start time and period of a bar the given point time should be part of More...
 
static DateTime GetNextDailyEndTime (Symbol symbol, DateTime exchangeTimeZoneDate, SecurityExchangeHours exchangeHours)
 Helper method to get the next daily end time, taking into account strict end times if appropriate More...
 
static bool OptionUseScaleFactor (Symbol symbol)
 Helper method that defines the types of options that should use scale factor More...
 
static bool UseStrictEndTime (bool dailyStrictEndTimeEnabled, Symbol symbol, TimeSpan increment, SecurityExchangeHours exchangeHours)
 Helper method to determine if we should use strict end time More...
 
static bool UseDailyStrictEndTimes (IAlgorithmSettings settings, BaseDataRequest request, Symbol symbol, TimeSpan increment, SecurityExchangeHours exchangeHours=null)
 Helper method to determine if we should use strict end time More...
 
static bool UseDailyStrictEndTimes (IAlgorithmSettings settings, Type dataType, Symbol symbol, TimeSpan increment, SecurityExchangeHours exchangeHours)
 Helper method to determine if we should use strict end time More...
 
static bool UseDailyStrictEndTimes (bool dailyStrictEndTimeEnabled, Type dataType, Symbol symbol, TimeSpan increment, SecurityExchangeHours exchangeHours)
 Helper method to determine if we should use strict end time More...
 
static bool UseDailyStrictEndTimes (Type dataType)
 True if this data type should use strict daily end times More...
 
static bool SetStrictEndTimes (IBaseData baseData, SecurityExchangeHours exchange)
 Helper method that if appropiate, will set the Time and EndTime of the given data point to it's daily strict times More...
 
static void ParseKey (string key, out string fileName, out string entryName)
 Helper to separate filename and entry from a given key for DataProviders More...
 

Static Public Attributes

static HashSet< string > SecurityTypeAsDataPath
 The different SecurityType used for data paths More...
 

Detailed Description

Provides methods for generating lean data file content

Definition at line 40 of file LeanData.cs.

Member Function Documentation

◆ GenerateLine() [1/2]

static string QuantConnect.Util.LeanData.GenerateLine ( IBaseData  data,
Resolution  resolution,
DateTimeZone  exchangeTimeZone,
DateTimeZone  dataTimeZone 
)
static

Converts the specified base data instance into a lean data file csv line. This method takes into account the fake that base data instances typically are time stamped in the exchange time zone, but need to be written to disk in the data time zone.

Definition at line 61 of file LeanData.cs.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ ParseTime()

static DateTime QuantConnect.Util.LeanData.ParseTime ( string  line,
DateTime  date,
Resolution  resolution 
)
static

Helper method that will parse a given data line in search of an associated date time

Definition at line 71 of file LeanData.cs.

◆ GenerateLine() [2/2]

static string QuantConnect.Util.LeanData.GenerateLine ( IBaseData  data,
SecurityType  securityType,
Resolution  resolution 
)
static

Converts the specified base data instance into a lean data file csv line

Definition at line 91 of file LeanData.cs.

◆ GetDataType()

static Type QuantConnect.Util.LeanData.GetDataType ( Resolution  resolution,
TickType  tickType 
)
static

Gets the data type required for the specified combination of resolution and tick type

Parameters
resolutionThe resolution, if Tick, the Type returned is always Tick
tickTypeThe TickType that primarily dictates the type returned
Returns
The Type used to create a subscription

Definition at line 489 of file LeanData.cs.

Here is the caller graph for this function:

◆ IsCommonLeanDataType()

static bool QuantConnect.Util.LeanData.IsCommonLeanDataType ( Type  baseDataType)
static

Determines if the Type is a 'common' type used throughout lean This method is helpful in creating SubscriptionDataConfig

Parameters
baseDataTypeThe Type to check
Returns
A bool indicating whether the type is of type TradeBar QuoteBar or OpenInterest

Definition at line 505 of file LeanData.cs.

Here is the caller graph for this function:

◆ IsValidConfiguration()

static bool QuantConnect.Util.LeanData.IsValidConfiguration ( SecurityType  securityType,
Resolution  resolution,
TickType  tickType 
)
static

Helper method to determine if a configuration set is valid

Definition at line 521 of file LeanData.cs.

Here is the caller graph for this function:

◆ GenerateZipFilePath() [1/2]

static string QuantConnect.Util.LeanData.GenerateZipFilePath ( string  dataDirectory,
Symbol  symbol,
DateTime  date,
Resolution  resolution,
TickType  tickType 
)
static

Generates the full zip file path rooted in the dataDirectory

Definition at line 533 of file LeanData.cs.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GenerateZipFilePath() [2/2]

static string QuantConnect.Util.LeanData.GenerateZipFilePath ( string  dataDirectory,
string  symbol,
SecurityType  securityType,
string  market,
DateTime  date,
Resolution  resolution 
)
static

Generates the full zip file path rooted in the dataDirectory

Definition at line 542 of file LeanData.cs.

Here is the call graph for this function:

◆ GenerateRelativeZipFileDirectory()

static string QuantConnect.Util.LeanData.GenerateRelativeZipFileDirectory ( Symbol  symbol,
Resolution  resolution 
)
static

Generates the relative zip directory for the specified symbol/resolution

Definition at line 550 of file LeanData.cs.

Here is the caller graph for this function:

◆ GenerateRelativeFactorFilePath()

static string QuantConnect.Util.LeanData.GenerateRelativeFactorFilePath ( Symbol  symbol)
static

Generates relative factor file paths for equities

Definition at line 598 of file LeanData.cs.

Here is the caller graph for this function:

◆ GenerateRelativeZipFilePath() [1/2]

static string QuantConnect.Util.LeanData.GenerateRelativeZipFilePath ( Symbol  symbol,
DateTime  date,
Resolution  resolution,
TickType  tickType 
)
static

Generates the relative zip file path rooted in the /Data directory

Definition at line 610 of file LeanData.cs.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GenerateRelativeZipFilePath() [2/2]

static string QuantConnect.Util.LeanData.GenerateRelativeZipFilePath ( string  symbol,
SecurityType  securityType,
string  market,
DateTime  date,
Resolution  resolution 
)
static

Generates the relative zip file path rooted in the /Data directory

Definition at line 618 of file LeanData.cs.

Here is the call graph for this function:

◆ GenerateRelativeUniversesDirectory()

static string QuantConnect.Util.LeanData.GenerateRelativeUniversesDirectory ( Symbol  symbol)
static

Generates the relative directory to the universe files for the specified symbol

Definition at line 632 of file LeanData.cs.

Here is the caller graph for this function:

◆ GenerateUniversesDirectory()

static string QuantConnect.Util.LeanData.GenerateUniversesDirectory ( string  dataDirectory,
Symbol  symbol 
)
static

Generates the directory to the universe files for the specified symbol

Definition at line 658 of file LeanData.cs.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GenerateZipEntryName()

static string QuantConnect.Util.LeanData.GenerateZipEntryName ( Symbol  symbol,
DateTime  date,
Resolution  resolution,
TickType  tickType 
)
static

Generate's the zip entry name to hold the specified data.

Definition at line 666 of file LeanData.cs.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GenerateZipFileName() [1/2]

static string QuantConnect.Util.LeanData.GenerateZipFileName ( Symbol  symbol,
DateTime  date,
Resolution  resolution,
TickType  tickType 
)
static

Generates the zip file name for the specified date of data.

Definition at line 792 of file LeanData.cs.

Here is the caller graph for this function:

◆ GenerateZipFileName() [2/2]

static string QuantConnect.Util.LeanData.GenerateZipFileName ( string  symbol,
SecurityType  securityType,
DateTime  date,
Resolution  resolution,
TickType tickType = null 
)
static

Creates the zip file name for a QC zip data file

Definition at line 857 of file LeanData.cs.

◆ GetCommonTickType()

static TickType QuantConnect.Util.LeanData.GetCommonTickType ( SecurityType  securityType)
static

Gets the tick type most commonly associated with the specified security type

Parameters
securityTypeThe security type
Returns
The most common tick type for the specified security type

Definition at line 886 of file LeanData.cs.

Here is the caller graph for this function:

◆ ReadSymbolFromZipEntry()

static Symbol QuantConnect.Util.LeanData.ReadSymbolFromZipEntry ( Symbol  symbol,
Resolution  resolution,
string  zipEntryName 
)
static

Creates a symbol from the specified zip entry name

Parameters
symbolThe root symbol of the output symbol
resolutionThe resolution of the data source producing the zip entry name
zipEntryNameThe zip entry name to be parsed
Returns
A new symbol representing the zip entry name

Definition at line 902 of file LeanData.cs.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GetCommonTickTypeForCommonDataTypes()

static TickType QuantConnect.Util.LeanData.GetCommonTickTypeForCommonDataTypes ( Type  type,
SecurityType  securityType 
)
static

Get the TickType for common Lean data types. If not a Lean common data type, return a TickType of Trade.

Parameters
typeA Type used to determine the TickType
securityTypeThe SecurityType used to determine the TickType
Returns
A TickType corresponding to the type

Definition at line 1012 of file LeanData.cs.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ ParseDataSecurityType()

static SecurityType QuantConnect.Util.LeanData.ParseDataSecurityType ( string  securityType)
static

Matches a data path security type with the SecurityType

This includes 'alternative'

Parameters
securityTypeThe data path security type
Returns
The matching security type for the given data path

Definition at line 1044 of file LeanData.cs.

Here is the caller graph for this function:

◆ TryParseSecurityType()

static bool QuantConnect.Util.LeanData.TryParseSecurityType ( string  fileName,
out SecurityType  securityType,
out string  market 
)
static

Parses file name into a Security and DateTime

Parameters
fileNameFile name to be parsed
securityTypeThe securityType as parsed from the fileName
marketThe market as parsed from the fileName

Definition at line 1059 of file LeanData.cs.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ TryParsePath() [1/2]

static bool QuantConnect.Util.LeanData.TryParsePath ( string  filePath,
out Symbol  symbol,
out DateTime  date,
out Resolution  resolution,
out TickType  tickType,
out Type  dataType 
)
static

Parses file name into a Security and DateTime

Parameters
filePathFile path to be parsed
symbolThe symbol as parsed from the fileName
dateDate of data in the file path. Only returned if the resolution is lower than Hourly
resolutionThe resolution of the symbol as parsed from the filePath
tickTypeThe tick type
dataTypeThe data type

Definition at line 1097 of file LeanData.cs.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ TryParsePath() [2/2]

static bool QuantConnect.Util.LeanData.TryParsePath ( string  fileName,
out Symbol  symbol,
out DateTime  date,
out Resolution  resolution 
)
static

Parses file name into a Security and DateTime

Parameters
fileNameFile name to be parsed
symbolThe symbol as parsed from the fileName
dateDate of data in the file path. Only returned if the resolution is lower than Hourly
resolutionThe resolution of the symbol as parsed from the filePath

Definition at line 1145 of file LeanData.cs.

Here is the call graph for this function:

◆ AggregateTradeBars()

static IEnumerable<TradeBar> QuantConnect.Util.LeanData.AggregateTradeBars ( IEnumerable< TradeBar bars,
Symbol  symbol,
TimeSpan  resolution 
)
static

Aggregates a list of second/minute bars at the requested resolution

Parameters
barsList of TradeBars
symbolSymbol of all tradeBars
resolutionDesired resolution for new TradeBars
Returns
List of aggregated TradeBars

Definition at line 1327 of file LeanData.cs.

◆ AggregateQuoteBars()

static IEnumerable<QuoteBar> QuantConnect.Util.LeanData.AggregateQuoteBars ( IEnumerable< QuoteBar bars,
Symbol  symbol,
TimeSpan  resolution 
)
static

Aggregates a list of second/minute bars at the requested resolution

Parameters
barsList of QuoteBars
symbolSymbol of all QuoteBars
resolutionDesired resolution for new QuoteBars
Returns
List of aggregated QuoteBars

Definition at line 1339 of file LeanData.cs.

◆ AggregateTicks()

static IEnumerable<QuoteBar> QuantConnect.Util.LeanData.AggregateTicks ( IEnumerable< Tick ticks,
Symbol  symbol,
TimeSpan  resolution 
)
static

Aggregates a list of ticks at the requested resolution

Parameters
ticksList of quote ticks
symbolSymbol of all ticks
resolutionDesired resolution for new QuoteBars
Returns
List of aggregated QuoteBars

Definition at line 1351 of file LeanData.cs.

◆ AggregateTicksToTradeBars()

static IEnumerable<TradeBar> QuantConnect.Util.LeanData.AggregateTicksToTradeBars ( IEnumerable< Tick ticks,
Symbol  symbol,
TimeSpan  resolution 
)
static

Aggregates a list of ticks at the requested resolution

Parameters
ticksList of trade ticks
symbolSymbol of all ticks
resolutionDesired resolution for new TradeBars
Returns
List of aggregated TradeBars

Definition at line 1363 of file LeanData.cs.

◆ GetDailyCalendar() [1/2]

static CalendarInfo QuantConnect.Util.LeanData.GetDailyCalendar ( DateTime  exchangeTimeZoneDate,
SecurityExchange  exchange,
bool  extendedMarketHours 
)
static

Helper method to return the start time and period of a bar the given point time should be part of

Parameters
exchangeTimeZoneDateThe point in time we want to get the bar information about
exchangeThe associated security exchange
extendedMarketHoursTrue if extended market hours should be taken into consideration
Returns
The calendar information that holds a start time and a period

Definition at line 1375 of file LeanData.cs.

Here is the caller graph for this function:

◆ GetDailyCalendar() [2/2]

static CalendarInfo QuantConnect.Util.LeanData.GetDailyCalendar ( DateTime  exchangeTimeZoneDate,
SecurityExchangeHours  exchangeHours,
bool  extendedMarketHours 
)
static

Helper method to return the start time and period of a bar the given point time should be part of

Parameters
exchangeTimeZoneDateThe point in time we want to get the bar information about
exchangeHoursThe associated exchange hours
extendedMarketHoursTrue if extended market hours should be taken into consideration
Returns
The calendar information that holds a start time and a period

Definition at line 1387 of file LeanData.cs.

Here is the call graph for this function:

◆ GetNextDailyEndTime()

static DateTime QuantConnect.Util.LeanData.GetNextDailyEndTime ( Symbol  symbol,
DateTime  exchangeTimeZoneDate,
SecurityExchangeHours  exchangeHours 
)
static

Helper method to get the next daily end time, taking into account strict end times if appropriate

Definition at line 1410 of file LeanData.cs.

Here is the call graph for this function:

◆ OptionUseScaleFactor()

static bool QuantConnect.Util.LeanData.OptionUseScaleFactor ( Symbol  symbol)
static

Helper method that defines the types of options that should use scale factor

Definition at line 1435 of file LeanData.cs.

Here is the caller graph for this function:

◆ UseStrictEndTime()

static bool QuantConnect.Util.LeanData.UseStrictEndTime ( bool  dailyStrictEndTimeEnabled,
Symbol  symbol,
TimeSpan  increment,
SecurityExchangeHours  exchangeHours 
)
static

Helper method to determine if we should use strict end time

Parameters
symbolThe associated symbol
incrementThe datas time increment

Definition at line 1446 of file LeanData.cs.

Here is the caller graph for this function:

◆ UseDailyStrictEndTimes() [1/4]

static bool QuantConnect.Util.LeanData.UseDailyStrictEndTimes ( IAlgorithmSettings  settings,
BaseDataRequest  request,
Symbol  symbol,
TimeSpan  increment,
SecurityExchangeHours  exchangeHours = null 
)
static

Helper method to determine if we should use strict end time

Definition at line 1462 of file LeanData.cs.

Here is the caller graph for this function:

◆ UseDailyStrictEndTimes() [2/4]

static bool QuantConnect.Util.LeanData.UseDailyStrictEndTimes ( IAlgorithmSettings  settings,
Type  dataType,
Symbol  symbol,
TimeSpan  increment,
SecurityExchangeHours  exchangeHours 
)
static

Helper method to determine if we should use strict end time

Definition at line 1471 of file LeanData.cs.

Here is the call graph for this function:

◆ UseDailyStrictEndTimes() [3/4]

static bool QuantConnect.Util.LeanData.UseDailyStrictEndTimes ( bool  dailyStrictEndTimeEnabled,
Type  dataType,
Symbol  symbol,
TimeSpan  increment,
SecurityExchangeHours  exchangeHours 
)
static

Helper method to determine if we should use strict end time

Definition at line 1479 of file LeanData.cs.

Here is the call graph for this function:

◆ UseDailyStrictEndTimes() [4/4]

static bool QuantConnect.Util.LeanData.UseDailyStrictEndTimes ( Type  dataType)
static

True if this data type should use strict daily end times

Definition at line 1487 of file LeanData.cs.

◆ SetStrictEndTimes()

static bool QuantConnect.Util.LeanData.SetStrictEndTimes ( IBaseData  baseData,
SecurityExchangeHours  exchange 
)
static

Helper method that if appropiate, will set the Time and EndTime of the given data point to it's daily strict times

Parameters
baseDataThe target data point
exchangeThe associated exchange hours

This method is used to set daily times on pre existing data, assuming it does not cover extended market hours

Definition at line 1499 of file LeanData.cs.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ ParseKey()

static void QuantConnect.Util.LeanData.ParseKey ( string  key,
out string  fileName,
out string  entryName 
)
static

Helper to separate filename and entry from a given key for DataProviders

Parameters
keyThe key to parse
fileNameFile name extracted
entryNameEntry name extracted

Definition at line 1538 of file LeanData.cs.

Here is the caller graph for this function:

Member Data Documentation

◆ SecurityTypeAsDataPath

HashSet<string> QuantConnect.Util.LeanData.SecurityTypeAsDataPath
static
Initial value:
=> Enum.GetNames(typeof(SecurityType))
.Select(x => x.ToLowerInvariant()).Union(new[] { "alternative" }).ToHashSet()

The different SecurityType used for data paths

This includes 'alternative'

Definition at line 52 of file LeanData.cs.


The documentation for this class was generated from the following file: