PAMI.highUtilitySpatialPattern.basic package
Submodules
PAMI.highUtilitySpatialPattern.basic.HDSHUIM module
- class PAMI.highUtilitySpatialPattern.basic.HDSHUIM.HDSHUIM(iFile: str, nFile: str, minUtil: int, sep: str = '\t')[source]
Bases:
_utilityPatterns- Description:
Spatial High Utility ItemSet Mining (SHUIM) [3] is an important model in data mining with many real-world applications. It involves finding all spatially interesting itemSets having high value in a quantitative spatio temporal database.
- Reference:
P. Pallikila et al., “Discovering Top-k Spatial High Utility Itemsets in Very Large Quantitative Spatiotemporal databases,” 2021 IEEE International Conference on Big Data (Big Data), Orlando, FL, USA, 2021, pp. 4925-4935, doi: 10.1109/BigData52589.2021.9671912.
- Parameters:
iFile – str : Name of the Input file to mine complete set of High Utility Spatial patterns
oFile – str : Name of the output file to store complete set of High Utility Spatial patterns
minSup – int or float or str : The user can specify minSup either in count or proportion of database size. If the program detects the data type of minSup is integer, then it treats minSup is expressed in count. Otherwise, it will be treated as float.
maxPer – float : The user can specify maxPer in count or proportion of database size. If the program detects the data type of maxPer is integer, then it treats maxPer is expressed in count.
minUtil – int : Minimum utility threshold given by User
nFile – str : Name of the input file to mine complete set of High Utility Spatial patterns
sep – str : This variable is used to distinguish items from one another in a transaction. The default seperator is tab space. However, the users can override their default separator.
- Attributes:
- iFilestr
Name of the input file to mine complete set of frequent patterns
- oFilestr
Name of the output file to store complete set of frequent patterns
- nFile: str
Name of Neighbourhood items file
- memoryRSSfloat
To store the total amount of RSS memory consumed by the program
- startTime:float
To record the start time of the mining process
- endTime:float
To record the completion time of the mining process
- minUtilint
The user given minUtil
- mapFMAP: list
EUCS map of the FHM algorithm
- candidates: int
candidates generated
- huiCnt: int
huis created
- neighbors: map
keep track of neighbours of elements
- mapOfPMU: map
a map to keep track of Probable Maximum utility(PMU) of each item
- Methods:
- mine()
Mining process will start from here
- getPatterns()
Complete set of patterns will be retrieved with this function
- save(oFile)
Complete set of frequent patterns will be loaded in to a output file
- constructCUL(x, compactUList, st, minUtil, length, exNeighbours)
A method to construct CUL’s database
- getPatternsAsDataFrame()
Complete set of frequent patterns will be loaded in to a dataframe
- getMemoryUSS()
Total amount of USS memory consumed by the mining process will be retrieved from this function
- getMemoryRSS()
Total amount of RSS memory consumed by the mining process will be retrieved from this function
- getRuntime()
Total amount of runtime taken by the mining process will be retrieved from this function
- Explore_SearchTree(prefix, uList, exNeighbours, minUtil)
A method to find all high utility itemSets
- updateClosed(x, compactUList, st, exCul, newT, ex, eyTs, length)
A method to update closed values
- saveItemSet(prefix, prefixLen, item, utility)
A method to save itemSets
- updateElement(z, compactUList, st, exCul, newT, ex, duPrevPos, eyTs)
A method to updates vales for duplicates
Executing the code on terminal:
Format: (.venv) $ python3 HDSHUIM.py <inputFile> <outputFile> <Neighbours> <minUtil> <separator> Example Usage: (.venv) $ python3 HDSHUIM.py sampleTDB.txt output.txt sampleN.txt 35 ','
Note
minSup will be considered in percentage of database transactions
Sample run of importing the code:
from PAMI.highUtilityGeoreferencedFrequentPattern.basic import HDSHUIM as alg obj=alg.HDSHUIM("input.txt","Neighbours.txt",35) obj.mine() Patterns = obj.getPatterns() print("Total number of Spatial High-Utility Patterns:", len(Patterns)) obj.save("output") memUSS = obj.getMemoryUSS() print("Total Memory in USS:", memUSS) memRSS = obj.getMemoryRSS() print("Total Memory in RSS", memRSS) run = obj.getRuntime() print("Total ExecutionTime in seconds:", run)
Credits:
The complete program was written by B.Sai Chitra under the supervision of Professor Rage Uday Kiran.
- getMemoryRSS() float[source]
Total amount of RSS memory consumed by the mining process will be retrieved from this function
- Returns:
returning RSS memory consumed by the mining process
- Return type:
float
- getMemoryUSS() float[source]
Total amount of USS memory consumed by the mining process will be retrieved from this function
- Returns:
returning USS memory consumed by the mining process
- Return type:
float
- getPatterns() Dict[str, str][source]
Function to send the set of frequent patterns after completion of the mining process
- Returns:
returning frequent patterns
- Return type:
dict
- getPatternsAsDataFrame() Dict[str, str][source]
Storing final frequent patterns in a dataframe
- Returns:
returning frequent patterns in a dataframe
- Return type:
pd.DataFrame
- getRuntime() float[source]
Calculating the total amount of runtime taken by the mining process
- Returns:
returning total amount of runtime taken by the mining process
- Return type:
float
PAMI.highUtilitySpatialPattern.basic.SHUIM module
- class PAMI.highUtilitySpatialPattern.basic.SHUIM.SHUIM(iFile: str, nFile: str, minUtil: int, sep: str = '\t')[source]
Bases:
_utilityPatterns- Description:
Spatial High Utility itemSet Mining (SHUIM) aims to discover all itemSets in a spatioTemporal database that satisfy the user-specified minimum utility and maximum distance constraints
- Reference:
Rage, Uday & Veena, Pamalla & Penugonda, Ravikumar & Raj, Bathala & Dao, Minh & Zettsu, Koji & Bommisetti, Sai. (2023). HDSHUI-miner: a novel algorithm for discovering spatial high-utility itemsets in high-dimensional spatiotemporal databases. Applied Intelligence. 53. 1-26. 10.1007/s10489-022-04436-w.
- Parameters:
iFile – str : Name of the Input file to mine complete set of High Utility Spatial patterns
oFile – str : Name of the output file to store complete set of High Utility Spatial patterns
minSup – int or float or str : The user can specify minSup either in count or proportion of database size. If the program detects the data type of minSup is integer, then it treats minSup is expressed in count. Otherwise, it will be treated as float.
maxPer – float : The user can specify maxPer in count or proportion of database size. If the program detects the data type of maxPer is integer, then it treats maxPer is expressed in count.
minUtil – int : Minimum utility threshold given by User
maxMemory – int : Maximum memory used by this program for running
candidateCount – int : Number of candidates to consider when calculating a high utility spatial pattern
nFile – str : Name of the input file to mine complete set of High Utility Spatial patterns
sep – str : This variable is used to distinguish items from one another in a transaction. The default seperator is tab space. However, the users can override their default separator.
- Attributes:
- iFilefile
Name of the input file to mine complete set of frequent patterns
- nFilefile
Name of the Neighbours file that contain neighbours of items
- oFilefile
Name of the output file to store complete set of frequent patterns
- memoryRSSfloat
To store the total amount of RSS memory consumed by the program
- startTime:float
To record the start time of the mining process
- endTime:float
To record the completion time of the mining process
- minUtilint
The user given minUtil
- highUtilityItemSets: map
set of high utility itemSets
- candidateCount: int
Number of candidates
- utilityBinArrayLU: list
A map to hold the pmu values of the items in database
- utilityBinArraySU: list
A map to hold the subtree utility values of the items is database
- oldNamesToNewNames: list
A map to hold the subtree utility values of the items is database
- newNamesToOldNames: list
A map to store the old name corresponding to new name
- Neighboursmap
A dictionary to store the neighbours of a item
maxMemory:Maximum memory used by this program for running patternCount: int
Number of SHUI’s
- itemsToKeep: list
keep only the promising items ie items having twu >= minUtil
- itemsToExplore: list
keep items that subtreeUtility grater than minUtil
- Methods:
- mine()
Mining process will start from here
- getPatterns()
Complete set of patterns will be retrieved with this function
- save(oFile)
Complete set of frequent patterns will be loaded in to a output file
- getPatternsAsDataFrame()
Complete set of frequent patterns will be loaded in to a dataframe
- getMemoryUSS()
Total amount of USS memory consumed by the mining process will be retrieved from this function
- getMemoryRSS()
Total amount of RSS memory consumed by the mining process will be retrieved from this function
- getRuntime()
Total amount of runtime taken by the mining process will be retrieved from this function
- calculateNeighbourIntersection(self, prefixLength)
A method to return common Neighbours of items
- backtrackingEFIM(transactionsOfP, itemsToKeep, itemsToExplore, prefixLength)
A method to mine the SHUIs Recursively
- useUtilityBinArraysToCalculateUpperBounds(transactionsPe, j, itemsToKeep, neighbourhoodList)
A method to calculate the sub-tree utility and local utility of all items that can extend itemSet P and e
- output(tempPosition, utility)
A method ave a high-utility itemSet to file or memory depending on what the user chose
- _isEqual(transaction1, transaction2)
A method to Check if two transaction are identical
- intersection(lst1, lst2)
A method that return the intersection of 2 list
- useUtilityBinArrayToCalculateSubtreeUtilityFirstTime(dataset)
Scan the initial database to calculate the subtree utility of each items using a utility-bin array
- sortDatabase(self, transactions)
A Method to sort transaction in the order of PMU
- sort_transaction(self, trans1, trans2)
A Method to sort transaction in the order of PMU
- useUtilityBinArrayToCalculateLocalUtilityFirstTime(self, dataset)
A method to scan the database using utility bin array to calculate the pmus
Executing the code on terminal:
Format: (.venv) $ python3 SHUIM.py <inputFile> <outputFile> <Neighbours> <minUtil> <sep> Example Usage: (.venv) $ python3 SHUIM.py sampleTDB.txt output.txt sampleN.txt 35
Note
minSup will be considered in percentage of database transactions
Sample run of importing the code:
from PAMI.highUtilitySpatialPattern.basic import SHUIM as alg obj=alg.SHUIM("input.txt","Neighbours.txt",35) obj.mine() frequentPatterns = obj.getPatterns() print("Total number of Spatial high utility Patterns:", len(frequentPatterns)) obj.save("output") memUSS = obj.getMemoryUSS() print("Total Memory in USS:", memUSS) memRSS = obj.getMemoryRSS() print("Total Memory in RSS", memRSS) run = obj.getRuntime() print("Total ExecutionTime in seconds:", run)
Credits:
The complete program was written by Pradeep Pallikila under the supervision of Professor Rage Uday Kiran.
- getMemoryRSS() float[source]
Total amount of RSS memory consumed by the mining process will be retrieved from this function
- Returns:
returning RSS memory consumed by the mining process
- Return type:
float
- getMemoryUSS() float[source]
Total amount of USS memory consumed by the mining process will be retrieved from this function
- Returns:
returning USS memory consumed by the mining process
- Return type:
float
- getPatterns() Dict[str, str][source]
Function to send the set of patterns after completion of the mining process
- Returns:
returning patterns
- Return type:
dict
- getPatternsAsDataFrame() DataFrame[source]
Storing final patterns in a dataframe
- Returns:
returning patterns in a dataframe
- Return type:
pd.DataFrame
- getRuntime() float[source]
Calculating the total amount of runtime taken by the mining process
- Returns:
returning total amount of runtime taken by the mining process
- Return type:
float