Provides methods for generating lean data file content More...

Static Public Member Functions
static string	GenerateLine (IBaseData data, Resolution resolution, DateTimeZone exchangeTimeZone, DateTimeZone dataTimeZone)
	Converts the specified base data instance into a lean data file csv line. This method takes into account the fake that base data instances typically are time stamped in the exchange time zone, but need to be written to disk in the data time zone. More...

static DateTime	ParseTime (string line, DateTime date, Resolution resolution)
	Helper method that will parse a given data line in search of an associated date time More...

static string	GenerateLine (IBaseData data, SecurityType securityType, Resolution resolution)
	Converts the specified base data instance into a lean data file csv line More...

static Type	GetDataType (Resolution resolution, TickType tickType)
	Gets the data type required for the specified combination of resolution and tick type More...

static bool	IsCommonLeanDataType (Type baseDataType)
	Determines if the Type is a 'common' type used throughout lean This method is helpful in creating SubscriptionDataConfig More...

static bool	IsValidConfiguration (SecurityType securityType, Resolution resolution, TickType tickType)
	Helper method to determine if a configuration set is valid More...

static string	GenerateZipFilePath (string dataDirectory, Symbol symbol, DateTime date, Resolution resolution, TickType tickType)
	Generates the full zip file path rooted in the dataDirectory More...

static string	GenerateZipFilePath (string dataDirectory, string symbol, SecurityType securityType, string market, DateTime date, Resolution resolution)
	Generates the full zip file path rooted in the dataDirectory More...

static string	GenerateRelativeZipFileDirectory (Symbol symbol, Resolution resolution)
	Generates the relative zip directory for the specified symbol/resolution More...

static string	GenerateRelativeFactorFilePath (Symbol symbol)
	Generates relative factor file paths for equities More...

static string	GenerateRelativeZipFilePath (Symbol symbol, DateTime date, Resolution resolution, TickType tickType)
	Generates the relative zip file path rooted in the /Data directory More...

static string	GenerateRelativeZipFilePath (string symbol, SecurityType securityType, string market, DateTime date, Resolution resolution)
	Generates the relative zip file path rooted in the /Data directory More...

static string	GenerateRelativeUniversesDirectory (Symbol symbol)
	Generates the relative directory to the universe files for the specified symbol More...

static string	GenerateUniversesDirectory (string dataDirectory, Symbol symbol)
	Generates the directory to the universe files for the specified symbol More...

static string	GenerateZipEntryName (Symbol symbol, DateTime date, Resolution resolution, TickType tickType)
	Generate's the zip entry name to hold the specified data. More...

static string	GenerateZipFileName (Symbol symbol, DateTime date, Resolution resolution, TickType tickType)
	Generates the zip file name for the specified date of data. More...

static string	GenerateZipFileName (string symbol, SecurityType securityType, DateTime date, Resolution resolution, TickType? tickType=null)
	Creates the zip file name for a QC zip data file More...

static TickType	GetCommonTickType (SecurityType securityType)
	Gets the tick type most commonly associated with the specified security type More...

static Symbol	ReadSymbolFromZipEntry (Symbol symbol, Resolution resolution, string zipEntryName)
	Creates a symbol from the specified zip entry name More...

static TickType	GetCommonTickTypeForCommonDataTypes (Type type, SecurityType securityType)
	Get the TickType for common Lean data types. If not a Lean common data type, return a TickType of Trade. More...

static SecurityType	ParseDataSecurityType (string securityType)
	Matches a data path security type with the SecurityType More...

static bool	TryParseSecurityType (string fileName, out SecurityType securityType, out string market)
	Parses file name into a Security and DateTime More...

static bool	TryParsePath (string filePath, out Symbol symbol, out DateTime date, out Resolution resolution, out TickType tickType, out Type dataType)
	Parses file name into a Security and DateTime More...

static bool	TryParsePath (string fileName, out Symbol symbol, out DateTime date, out Resolution resolution)
	Parses file name into a Security and DateTime More...

static IEnumerable< TradeBar >	AggregateTradeBars (IEnumerable< TradeBar > bars, Symbol symbol, TimeSpan resolution)
	Aggregates a list of second/minute bars at the requested resolution More...

static IEnumerable< QuoteBar >	AggregateQuoteBars (IEnumerable< QuoteBar > bars, Symbol symbol, TimeSpan resolution)
	Aggregates a list of second/minute bars at the requested resolution More...

static IEnumerable< QuoteBar >	AggregateTicks (IEnumerable< Tick > ticks, Symbol symbol, TimeSpan resolution)
	Aggregates a list of ticks at the requested resolution More...

static IEnumerable< TradeBar >	AggregateTicksToTradeBars (IEnumerable< Tick > ticks, Symbol symbol, TimeSpan resolution)
	Aggregates a list of ticks at the requested resolution More...

static CalendarInfo	GetDailyCalendar (DateTime exchangeTimeZoneDate, SecurityExchange exchange, bool extendedMarketHours)
	Helper method to return the start time and period of a bar the given point time should be part of More...

static CalendarInfo	GetDailyCalendar (DateTime exchangeTimeZoneDate, SecurityExchangeHours exchangeHours, bool extendedMarketHours)
	Helper method to return the start time and period of a bar the given point time should be part of More...

static DateTime	GetNextDailyEndTime (Symbol symbol, DateTime exchangeTimeZoneDate, SecurityExchangeHours exchangeHours)
	Helper method to get the next daily end time, taking into account strict end times if appropriate More...

static bool	OptionUseScaleFactor (Symbol symbol)
	Helper method that defines the types of options that should use scale factor More...

static bool	UseStrictEndTime (bool dailyStrictEndTimeEnabled, Symbol symbol, TimeSpan increment, SecurityExchangeHours exchangeHours)
	Helper method to determine if we should use strict end time More...

static bool	UseDailyStrictEndTimes (IAlgorithmSettings settings, BaseDataRequest request, Symbol symbol, TimeSpan increment)
	Helper method to determine if we should use strict end time More...

static bool	UseDailyStrictEndTimes (IAlgorithmSettings settings, Type dataType, Symbol symbol, TimeSpan increment, SecurityExchangeHours exchangeHours)
	Helper method to determine if we should use strict end time More...

static bool	UseDailyStrictEndTimes (Type dataType)
	True if this data type should use strict daily end times More...

static bool	SetStrictEndTimes (IBaseData baseData, SecurityExchangeHours exchange)
	Helper method that if appropiate, will set the Time and EndTime of the given data point to it's daily strict times More...

static void	ParseKey (string key, out string fileName, out string entryName)
	Helper to separate filename and entry from a given key for DataProviders More...

Static Public Attributes
static HashSet< string >	SecurityTypeAsDataPath
	The different SecurityType used for data paths More...

Detailed Description

Provides methods for generating lean data file content

Definition at line 40 of file LeanData.cs.

Member Function Documentation

◆ GenerateLine() [1/2]

static string QuantConnect.Util.LeanData.GenerateLine	(	IBaseData	data,
		Resolution	resolution,
		DateTimeZone	exchangeTimeZone,
		DateTimeZone	dataTimeZone
	)

static

Converts the specified base data instance into a lean data file csv line. This method takes into account the fake that base data instances typically are time stamped in the exchange time zone, but need to be written to disk in the data time zone.

Definition at line 61 of file LeanData.cs.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ ParseTime()

static DateTime QuantConnect.Util.LeanData.ParseTime	(	string	line,
		DateTime	date,
		Resolution	resolution
	)

static

Helper method that will parse a given data line in search of an associated date time

Definition at line 71 of file LeanData.cs.

◆ GenerateLine() [2/2]

static string QuantConnect.Util.LeanData.GenerateLine	(	IBaseData	data,
		SecurityType	securityType,
		Resolution	resolution
	)

static

Converts the specified base data instance into a lean data file csv line

Definition at line 91 of file LeanData.cs.

◆ GetDataType()

static Type QuantConnect.Util.LeanData.GetDataType	(	Resolution	resolution,
		TickType	tickType
	)

static

Gets the data type required for the specified combination of resolution and tick type

Parameters

resolution	The resolution, if Tick, the Type returned is always Tick
tickType	The TickType that primarily dictates the type returned

Returns: The Type used to create a subscription

Definition at line 489 of file LeanData.cs.

Here is the caller graph for this function:

◆ IsCommonLeanDataType()

static bool QuantConnect.Util.LeanData.IsCommonLeanDataType ( Type baseDataType )

static

Determines if the Type is a 'common' type used throughout lean This method is helpful in creating SubscriptionDataConfig

Parameters

baseDataType The Type to check

Returns: A bool indicating whether the type is of type TradeBar QuoteBar or OpenInterest

Definition at line 505 of file LeanData.cs.

Here is the caller graph for this function:

◆ IsValidConfiguration()

static bool QuantConnect.Util.LeanData.IsValidConfiguration	(	SecurityType	securityType,
		Resolution	resolution,
		TickType	tickType
	)

static

Helper method to determine if a configuration set is valid

Definition at line 521 of file LeanData.cs.

Here is the caller graph for this function:

◆ GenerateZipFilePath() [1/2]

static string QuantConnect.Util.LeanData.GenerateZipFilePath	(	string	dataDirectory,
		Symbol	symbol,
		DateTime	date,
		Resolution	resolution,
		TickType	tickType
	)

static

Generates the full zip file path rooted in the dataDirectory

Definition at line 533 of file LeanData.cs.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ GenerateZipFilePath() [2/2]

static string QuantConnect.Util.LeanData.GenerateZipFilePath	(	string	dataDirectory,
		string	symbol,
		SecurityType	securityType,
		string	market,
		DateTime	date,
		Resolution	resolution
	)

static

Generates the full zip file path rooted in the dataDirectory

Definition at line 542 of file LeanData.cs.

Here is the call graph for this function:

◆ GenerateRelativeZipFileDirectory()

static string QuantConnect.Util.LeanData.GenerateRelativeZipFileDirectory	(	Symbol	symbol,
		Resolution	resolution
	)

static

Generates the relative zip directory for the specified symbol/resolution

Definition at line 550 of file LeanData.cs.

Here is the caller graph for this function:

◆ GenerateRelativeFactorFilePath()

static string QuantConnect.Util.LeanData.GenerateRelativeFactorFilePath ( Symbol symbol )

static

Generates relative factor file paths for equities

Definition at line 598 of file LeanData.cs.

Here is the caller graph for this function:

◆ GenerateRelativeZipFilePath() [1/2]

static string QuantConnect.Util.LeanData.GenerateRelativeZipFilePath	(	Symbol	symbol,
		DateTime	date,
		Resolution	resolution,
		TickType	tickType
	)

static

Generates the relative zip file path rooted in the /Data directory

Definition at line 610 of file LeanData.cs.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ GenerateRelativeZipFilePath() [2/2]

static string QuantConnect.Util.LeanData.GenerateRelativeZipFilePath	(	string	symbol,
		SecurityType	securityType,
		string	market,
		DateTime	date,
		Resolution	resolution
	)

static

Generates the relative zip file path rooted in the /Data directory

Definition at line 618 of file LeanData.cs.

Here is the call graph for this function:

◆ GenerateRelativeUniversesDirectory()

static string QuantConnect.Util.LeanData.GenerateRelativeUniversesDirectory ( Symbol symbol )

static

Generates the relative directory to the universe files for the specified symbol

Definition at line 632 of file LeanData.cs.

Here is the caller graph for this function:

◆ GenerateUniversesDirectory()

static string QuantConnect.Util.LeanData.GenerateUniversesDirectory	(	string	dataDirectory,
		Symbol	symbol
	)

static

Generates the directory to the universe files for the specified symbol

Definition at line 658 of file LeanData.cs.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ GenerateZipEntryName()

static string QuantConnect.Util.LeanData.GenerateZipEntryName	(	Symbol	symbol,
		DateTime	date,
		Resolution	resolution,
		TickType	tickType
	)

static

Generate's the zip entry name to hold the specified data.

Definition at line 666 of file LeanData.cs.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ GenerateZipFileName() [1/2]

static string QuantConnect.Util.LeanData.GenerateZipFileName	(	Symbol	symbol,
		DateTime	date,
		Resolution	resolution,
		TickType	tickType
	)

static

Generates the zip file name for the specified date of data.

Definition at line 792 of file LeanData.cs.

Here is the caller graph for this function:

◆ GenerateZipFileName() [2/2]

static string QuantConnect.Util.LeanData.GenerateZipFileName	(	string	symbol,
		SecurityType	securityType,
		DateTime	date,
		Resolution	resolution,
		TickType?	tickType = `null`
	)

static

Creates the zip file name for a QC zip data file

Definition at line 857 of file LeanData.cs.

◆ GetCommonTickType()

static TickType QuantConnect.Util.LeanData.GetCommonTickType ( SecurityType securityType )

static

Gets the tick type most commonly associated with the specified security type

Parameters

securityType The security type

Returns: The most common tick type for the specified security type

Definition at line 886 of file LeanData.cs.

Here is the caller graph for this function:

◆ ReadSymbolFromZipEntry()

static Symbol QuantConnect.Util.LeanData.ReadSymbolFromZipEntry	(	Symbol	symbol,
		Resolution	resolution,
		string	zipEntryName
	)

static

Creates a symbol from the specified zip entry name

Parameters

symbol	The root symbol of the output symbol
resolution	The resolution of the data source producing the zip entry name
zipEntryName	The zip entry name to be parsed

Returns: A new symbol representing the zip entry name

Definition at line 902 of file LeanData.cs.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ GetCommonTickTypeForCommonDataTypes()

static TickType QuantConnect.Util.LeanData.GetCommonTickTypeForCommonDataTypes	(	Type	type,
		SecurityType	securityType
	)

static

Get the TickType for common Lean data types. If not a Lean common data type, return a TickType of Trade.

Parameters

type	A Type used to determine the TickType
securityType	The SecurityType used to determine the TickType

Returns: A TickType corresponding to the type

Definition at line 1012 of file LeanData.cs.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ ParseDataSecurityType()

static SecurityType QuantConnect.Util.LeanData.ParseDataSecurityType ( string securityType )

static

Matches a data path security type with the SecurityType

This includes 'alternative'

Parameters

securityType The data path security type

Returns: The matching security type for the given data path

Definition at line 1044 of file LeanData.cs.

Here is the caller graph for this function:

◆ TryParseSecurityType()

static bool QuantConnect.Util.LeanData.TryParseSecurityType	(	string	fileName,
		out SecurityType	securityType,
		out string	market
	)

static

Parses file name into a Security and DateTime

Parameters

fileName	File name to be parsed
securityType	The securityType as parsed from the fileName
market	The market as parsed from the fileName

Definition at line 1059 of file LeanData.cs.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ TryParsePath() [1/2]

static bool QuantConnect.Util.LeanData.TryParsePath	(	string	filePath,
		out Symbol	symbol,
		out DateTime	date,
		out Resolution	resolution,
		out TickType	tickType,
		out Type	dataType
	)

static

Parses file name into a Security and DateTime

Parameters

filePath	File path to be parsed
symbol	The symbol as parsed from the fileName
date	Date of data in the file path. Only returned if the resolution is lower than Hourly
resolution	The resolution of the symbol as parsed from the filePath
tickType	The tick type
dataType	The data type

Definition at line 1097 of file LeanData.cs.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ TryParsePath() [2/2]

static bool QuantConnect.Util.LeanData.TryParsePath	(	string	fileName,
		out Symbol	symbol,
		out DateTime	date,
		out Resolution	resolution
	)

static

Parses file name into a Security and DateTime

Parameters

fileName	File name to be parsed
symbol	The symbol as parsed from the fileName
date	Date of data in the file path. Only returned if the resolution is lower than Hourly
resolution	The resolution of the symbol as parsed from the filePath

Definition at line 1145 of file LeanData.cs.

Here is the call graph for this function:

◆ AggregateTradeBars()

static IEnumerable<TradeBar> QuantConnect.Util.LeanData.AggregateTradeBars	(	IEnumerable< TradeBar >	bars,
		Symbol	symbol,
		TimeSpan	resolution
	)

static

Aggregates a list of second/minute bars at the requested resolution

Parameters

bars	List of TradeBars
symbol	Symbol of all tradeBars
resolution	Desired resolution for new TradeBars

Returns: List of aggregated TradeBars

Definition at line 1327 of file LeanData.cs.

◆ AggregateQuoteBars()

static IEnumerable<QuoteBar> QuantConnect.Util.LeanData.AggregateQuoteBars	(	IEnumerable< QuoteBar >	bars,
		Symbol	symbol,
		TimeSpan	resolution
	)

static

Aggregates a list of second/minute bars at the requested resolution

Parameters

bars	List of QuoteBars
symbol	Symbol of all QuoteBars
resolution	Desired resolution for new QuoteBars

Returns: List of aggregated QuoteBars

Definition at line 1339 of file LeanData.cs.

◆ AggregateTicks()

static IEnumerable<QuoteBar> QuantConnect.Util.LeanData.AggregateTicks	(	IEnumerable< Tick >	ticks,
		Symbol	symbol,
		TimeSpan	resolution
	)

static

Aggregates a list of ticks at the requested resolution

Parameters

ticks	List of quote ticks
symbol	Symbol of all ticks
resolution	Desired resolution for new QuoteBars

Returns: List of aggregated QuoteBars

Definition at line 1351 of file LeanData.cs.

◆ AggregateTicksToTradeBars()

static IEnumerable<TradeBar> QuantConnect.Util.LeanData.AggregateTicksToTradeBars	(	IEnumerable< Tick >	ticks,
		Symbol	symbol,
		TimeSpan	resolution
	)

static

Aggregates a list of ticks at the requested resolution

Parameters

ticks	List of trade ticks
symbol	Symbol of all ticks
resolution	Desired resolution for new TradeBars

Returns: List of aggregated TradeBars

Definition at line 1363 of file LeanData.cs.

◆ GetDailyCalendar() [1/2]

static CalendarInfo QuantConnect.Util.LeanData.GetDailyCalendar	(	DateTime	exchangeTimeZoneDate,
		SecurityExchange	exchange,
		bool	extendedMarketHours
	)

static

Helper method to return the start time and period of a bar the given point time should be part of

Parameters

exchangeTimeZoneDate	The point in time we want to get the bar information about
exchange	The associated security exchange
extendedMarketHours	True if extended market hours should be taken into consideration

Returns: The calendar information that holds a start time and a period

Definition at line 1375 of file LeanData.cs.

Here is the caller graph for this function:

◆ GetDailyCalendar() [2/2]

static CalendarInfo QuantConnect.Util.LeanData.GetDailyCalendar	(	DateTime	exchangeTimeZoneDate,
		SecurityExchangeHours	exchangeHours,
		bool	extendedMarketHours
	)

static

Helper method to return the start time and period of a bar the given point time should be part of

Parameters

exchangeTimeZoneDate	The point in time we want to get the bar information about
exchangeHours	The associated exchange hours
extendedMarketHours	True if extended market hours should be taken into consideration

Returns: The calendar information that holds a start time and a period

Definition at line 1387 of file LeanData.cs.

Here is the call graph for this function:

◆ GetNextDailyEndTime()

static DateTime QuantConnect.Util.LeanData.GetNextDailyEndTime	(	Symbol	symbol,
		DateTime	exchangeTimeZoneDate,
		SecurityExchangeHours	exchangeHours
	)

static

Helper method to get the next daily end time, taking into account strict end times if appropriate

Definition at line 1410 of file LeanData.cs.

Here is the call graph for this function:

◆ OptionUseScaleFactor()

static bool QuantConnect.Util.LeanData.OptionUseScaleFactor ( Symbol symbol )

static

Helper method that defines the types of options that should use scale factor

Definition at line 1435 of file LeanData.cs.

Here is the caller graph for this function:

◆ UseStrictEndTime()

static bool QuantConnect.Util.LeanData.UseStrictEndTime	(	bool	dailyStrictEndTimeEnabled,
		Symbol	symbol,
		TimeSpan	increment,
		SecurityExchangeHours	exchangeHours
	)

static

Helper method to determine if we should use strict end time

Parameters

symbol	The associated symbol
increment	The datas time increment

Definition at line 1446 of file LeanData.cs.

Here is the caller graph for this function:

◆ UseDailyStrictEndTimes() [1/3]

static bool QuantConnect.Util.LeanData.UseDailyStrictEndTimes	(	IAlgorithmSettings	settings,
		BaseDataRequest	request,
		Symbol	symbol,
		TimeSpan	increment
	)

static

Helper method to determine if we should use strict end time

Definition at line 1462 of file LeanData.cs.

Here is the caller graph for this function:

◆ UseDailyStrictEndTimes() [2/3]

static bool QuantConnect.Util.LeanData.UseDailyStrictEndTimes	(	IAlgorithmSettings	settings,
		Type	dataType,
		Symbol	symbol,
		TimeSpan	increment,
		SecurityExchangeHours	exchangeHours
	)

static

Helper method to determine if we should use strict end time

Definition at line 1470 of file LeanData.cs.

Here is the call graph for this function:

◆ UseDailyStrictEndTimes() [3/3]

static bool QuantConnect.Util.LeanData.UseDailyStrictEndTimes ( Type dataType )

static

True if this data type should use strict daily end times

Definition at line 1478 of file LeanData.cs.

◆ SetStrictEndTimes()

static bool QuantConnect.Util.LeanData.SetStrictEndTimes	(	IBaseData	baseData,
		SecurityExchangeHours	exchange
	)

static

Helper method that if appropiate, will set the Time and EndTime of the given data point to it's daily strict times

Parameters

baseData	The target data point
exchange	The associated exchange hours

This method is used to set daily times on pre existing data, assuming it does not cover extended market hours

Definition at line 1490 of file LeanData.cs.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ ParseKey()

static void QuantConnect.Util.LeanData.ParseKey	(	string	key,
		out string	fileName,
		out string	entryName
	)

static

Helper to separate filename and entry from a given key for DataProviders

Parameters

key	The key to parse
fileName	File name extracted
entryName	Entry name extracted

Definition at line 1529 of file LeanData.cs.

Here is the caller graph for this function:

Member Data Documentation

◆ SecurityTypeAsDataPath

HashSet<string> QuantConnect.Util.LeanData.SecurityTypeAsDataPath

static

Initial value:

=> Enum.GetNames(typeof(SecurityType))

.Select(x => x.ToLowerInvariant()).Union(new[] { "alternative" }).ToHashSet()

The different SecurityType used for data paths

This includes 'alternative'

Definition at line 52 of file LeanData.cs.

The documentation for this class was generated from the following file:

Common/Util/LeanData.cs

Static Public Member Functions

Static Public Attributes

Detailed Description

Member Function Documentation

◆ GenerateLine() [1/2]

◆ ParseTime()

◆ GenerateLine() [2/2]

◆ GetDataType()

◆ IsCommonLeanDataType()

◆ IsValidConfiguration()

◆ GenerateZipFilePath() [1/2]

◆ GenerateZipFilePath() [2/2]

◆ GenerateRelativeZipFileDirectory()

◆ GenerateRelativeFactorFilePath()

◆ GenerateRelativeZipFilePath() [1/2]

◆ GenerateRelativeZipFilePath() [2/2]

◆ GenerateRelativeUniversesDirectory()

◆ GenerateUniversesDirectory()

◆ GenerateZipEntryName()

◆ GenerateZipFileName() [1/2]

◆ GenerateZipFileName() [2/2]

◆ GetCommonTickType()

◆ ReadSymbolFromZipEntry()

◆ GetCommonTickTypeForCommonDataTypes()

◆ ParseDataSecurityType()

◆ TryParseSecurityType()

◆ TryParsePath() [1/2]

◆ TryParsePath() [2/2]

◆ AggregateTradeBars()

◆ AggregateQuoteBars()

◆ AggregateTicks()

◆ AggregateTicksToTradeBars()

◆ GetDailyCalendar() [1/2]

◆ GetDailyCalendar() [2/2]

◆ GetNextDailyEndTime()

◆ OptionUseScaleFactor()

◆ UseStrictEndTime()

◆ UseDailyStrictEndTimes() [1/3]

◆ UseDailyStrictEndTimes() [2/3]

◆ UseDailyStrictEndTimes() [3/3]

◆ SetStrictEndTimes()

◆ ParseKey()

Member Data Documentation

◆ SecurityTypeAsDataPath