scripts.core
Author: Carmen Meinson
- class Model
Bases:
object
- add_module(module_name: str, module: scripts.core.module.Module) None
- get_module_names() Set[str]
- Returns
names of all modules that are currently active in the model
- Return type
Set[str]
- switch_events(events_to_remove: Set[str], events_to_add: Dict[str, scripts.core.gesture_event.GestureEvent]) None
First removes the specified events from the model and adds new ones in afterwards. Removing an event may also lead to removing of some gestures that are no longer used by any events in the model. If all gestures from one module get removed so does the module.
- Parameters
events_to_remove (Set[str]) – names of the events to remove
events_to_add (Dict[str, GestureEvent]) – names of the events to add mapped to the event instances
- add_gesture(module_name: str, gesture_name: str, primitives: Set[scripts.core.gesture_factory.Primitive]) None
Add a gesture to the given module. If the gesture is already in the module then nothing is done. (this means that if there is an attempt to add a new gesture by the same name as a previous one, it will be ignored)
- Parameters
module_name (str) – module to which to add the gesture to
gesture_name (str) – name of the gesture to be added
primitives (Set[Primitive]) – primitives that describe the gesture
- Raises
RuntimeError – if the module we are trying to add the gesture to is not in the model
- remove_gesture(gesture_name: str) None
Simply removes the gesture from the model so that no new instances of this gesture will be created. Any currently active instances of this gesture will not be deactivated (are expected to deactivate naturally) Does not check if any events were using it so removing a gesture in use may lead to an event that cant be activated
- Parameters
gesture_name (str) – name of the gesture to remove
- Raises
RuntimeError – if a gesture with the given name has not been added to the model
- process_frame(frame: numpy.ndarray) scripts.core.raw_data.RawData
- Process the frame in following steps:
Process the frame in each module, from which we obtain the frames raw data and new Gesture instances may be created.
Activate in the model each Gesture that was created from the modules, which may (de)activate of some gesture events
Update all the currently active gestures, which may result in deactivation of some of them, and consequently (de)activation of some gesture events
Run all active gesture events, which may result in the trigger functions in the events being called.
- Parameters
frame (ndarray) – image reflecting the frame
- Returns
Coordinates of all the landmarks detected in the frame
- Return type
Author: Carmen Meinson
- class LandmarkDetector
Bases:
object
Interface for each modules landmark detector
- get_raw_data(raw_data: scripts.core.raw_data.RawData, image: numpy.ndarray) None
Adds the xy(z) coordinates of all the landmarks detected on the image into the RawData instance.
- Parameters
raw_data (RawData) – RawData instance to add the landmarks to
image (ndarray) – Image to process with mediapipe and read the landmarks locations from
- class Module
Bases:
object
- update_and_get_activated_gestures(frame_data: scripts.core.raw_data.RawData, image: numpy.ndarray) Set[scripts.core.gesture.Gesture]
Use the modules landmark detector (ML library) to retrieve the RawData (aka the coordinates of all landmarks) from the frame.
Update all position trackers with the RawData, which results in a set of primitives that had changed since the last frame.
Update all the gesture factories that use the changed primitives, which may create new active Gestures.
- add_gesture(gesture_name: str, primitives: Set[scripts.core.gesture_factory.Primitive]) None
Adds a new available gesture to the module.
- Parameters
gesture_name (str) – name of the gesture to be added
primitives (Set[Primitive]) – primitives that describe the gesture
- remove_gesture(gesture_name: str) None
Remove the gesture from the module. May lead to removal of some primitives (if the primitive is not used by any gestures anymore)
- Parameters
gesture_name (str) – name of the gesture to remove
- get_currently_used_primitives() Set[str]
Get names of all primitives that are used by any of the available gestures.
- Returns
names of currently used primitives
- Return type
Set[str]
- reset() None
Resets all position trackers
- classmethod calibrate() None
- classmethod pre_initialize() None
Can be called to perform all of the time consuming setup of the Module before initialization. If is not called then all the setup is done when the Module is first initialized.
Author: Carmen Meinson
- class RawData
Bases:
object
- add_landmark(bodypart_name: str, landmark_name: str, coordinates: numpy.ndarray) None
- combine(raw_data: scripts.core.raw_data.RawData) None
Combine the data from 2 RawData instances. Note the instances MUST contain different body parts.
- Parameters
raw_data (RawData) – RawData instance the data of which should be copied over to this one
- Raises
RuntimeError – raised if the instances do contain the same body parts
- get_bodyparts() Set[str]
- Returns
names of all the body parts that have been added to the RawData instance
- Return type
Set[str]
- get_data(bodypart_name: str) Optional[Dict[str, numpy.ndarray]]
Returns the data of the body part, if it has been added (aka if it has been detected in the frame)
- Parameters
bodypart_name (str) – bodypart, the data of which should be returned. e.g. “body” or “Left”
- Returns
Dictionary of all the landmarks added to the specific body part with the name of the landmark as key and the coordinates [x,y(,z)] as value
- Return type
Optional[Dict[str, np.ndarray]]
- get_landmark(bodypart_name: str, landmark_name: str) Optional[numpy.ndarray]
Returns the coordinates of the landmark, if it has been added to the specified body part (aka if it has been detected in the frame)
- Parameters
bodypart_name (str) – body part that the specified landmark should belong to e.g. “body” or “Left”
landmark_name (str) – landmark, the coordinate of which should be returned. e.g. “index_tip” or “wrist”
- Returns
coordinate of the landmark [x,y(,z)]
- Return type
Optional[np.ndarray]
Author: Carmen Meinson
- class Position(raw_data: Dict[str, numpy.ndarray], used_primitives: Optional[Set[str]] = None)
Bases:
object
- get_primitive(name: str) Optional[bool]
Returns the state of the given primitive in the Position if it has been calcualated. Returns None If the primitive has not been calculated, hence if it either is not defined for current module or the land marks needed to calculate it were not provided
- Parameters
name (str) – name of the primitive (e.g. “palm_facing_camera”, “index_pinched” or “punchleft”)
- Returns
state of the primitive
- Return type
Optional[bool]
- get_landmark(name: str) Optional[numpy.ndarray]
Returns the coordinates of the landmark. Returns None if the landmarks was not provided to the Position on initialization
- Parameters
name (str) – name of the landmark
- Returns
coordinates of the landmark [x,y(,z)]
- Return type
Optional[np.ndarray]
- get_primitives_names() Set[str]
- Returns
Names of all primitives that are calculated by the specific modules Position class on initialization
- Return type
List[str]
Author: Carmen Meinson
- class PositionTracker(name: str, position_class: Type[scripts.core.position.Position])
Bases:
object
- update(raw_data: scripts.core.raw_data.RawData, used_primitives: Optional[Set[str]] = None) Set[str]
Creates a Position instance out of the provided RawData and compares the state of all the primitives to the previous Position. Returns the list of all the changed primitives. If there was no previous position (so if it is the first update() call or if the tracker has been reset) all primitives are considered to have changed.
If used_primitives is not None the Position class may only calculate the primitives given. If it is None all primitives are calculated
- Parameters
raw_data (RawData) – RawData instance containing all the landmarks coordinates detected in the frame
used_primitives (Set[str]) – names of all the primitives that are currently used by the module
- Returns
set of changed primitives since the last update() call
- Return type
Set[str]
- get_name() str
returns the name of the tracker thus also reflecting the name of the body part
- get_current_position() Optional[scripts.core.position.Position]
Returns the Position instance generated by the last update() call. If there has been no such call or if the tracker has been reset returns None
- Returns
last Position instance
- Return type
Optional[Position]
- reset() Set[str]
Resets the Tracker instance to the initial state and returns all the changed primitives in doing so. Thus the current position is again None and by default all primitives are considered to have changed (unless the tracker was already in the reset state, in which case none of the primitives are considered to have changed)
- Returns
set of changed primitives
- Return type
Set[str]
Author: Carmen Meinson
- class Primitive(name, state)
Bases:
tuple
- name
Alias for field number 0
- state
Alias for field number 1
- class GestureFactory(name: str, gesture_class: Type[scripts.core.gesture.Gesture], primitives: Set[scripts.core.gesture_factory.Primitive])
Bases:
object
- update(tracker: scripts.core.position_tracker.PositionTracker) Optional[scripts.core.gesture.Gesture]
Creates an instance of the gesture it represents if the current position in the provided tracker matches the gesture criteria (if all the primitives of interest are in needed states)
- Parameters
tracker (PositionTracker) – tracker containing the current position to be checked
- Returns
a new gesture instance if such is created
- Return type
Optional[Gesture]
- get_name() str
Author: Carmen Meinson
- class Gesture(name: str, position_checker: Callable[[scripts.core.position.Position], bool], tracker: scripts.core.position_tracker.PositionTracker, max_pos_q_size: Optional[int] = None)
Bases:
object
- update() bool
Records the current Position of the body part this Gesture was created from. With that Position checks if the Gesture still remains active (if all the states of the primitives of interest are in the required states) (This function is intended to be called each frame until the Gesture deactivates)
- Returns
The current state of the Gesture
- Return type
bool
- get_name() str
Returns name of represented gesture.
- Returns
name of represented gesture
- Return type
str
- get_bodypart_name() str
Returns name of the body part of the represented gesture.
- Returns
name of body part
- Return type
str
- get_frames_held() int
Note the number of positions stored and retrievable from the gesture instance may be less then the number of frames held in case the max_pos_q_size has been reached.
- Returns
number of frames the gesture as been held for
- Return type
int
- get_frames_stored() int
Note the number of positions stored and retrievable from the gesture instance may be less then the number of frames held in case the max_pos_q_size has been reached.
- Returns
number of positions stored in the gesture instance
- Return type
int
- get_last_position() scripts.core.position.Position
- get_last_n_positions(n: int) List[scripts.core.position.Position]
- get_time_held() float
Author: Carmen Meinson
- class GestureEvent(gesture_types: Set[str], event_trigger_types: Set[str], bodypart_types: Set[str])
Bases:
object
- set_trigger_functions(funcs: Dict[str, Callable]) None
- set_bodypart_names(bodypart_names) None
- set_up() None
Called right after the event is added to the model - so after the trigger functions and bodypart names have been set. The method could be used to for example set up the display elements.
- update() None
Checks the event firing conditions and then may call according trigger functions. (the function is expected to be called each frame until the state of the GestureEvent becomes False)
- get_state() bool
- force_deactivate() None
As some events need to finish some processing before being deactivated, they can implement this method. It is called when the events are removed from the model. (e.g. for example if an event has called the mouse press function it needs to release the mouse before deactivation)
- notify_gesture_activated(gesture: scripts.core.gesture.Gesture) None
Notifies that a new gesture of the same type that is used by the GestureEvent has activated. If the activated gesture has the correct body part name and gesture type combination, it is stored in the GestureEvent instance. The state of the GestureEvent is then checked in case the activation of the gesture changed it.
- Parameters
gesture (Gesture) – the gesture that activated
- notify_gesture_deactivated(gesture: scripts.core.gesture.Gesture) None
Notifies that a new gesture of the same type that is used by the GestureEvent has deactivated. If the deactivated gesture was previously stored in the GestureEvent instance it is now removed (although there may be cases where it is kept to perform calculations later). The state of the GestureEvent is then checked in case the deactivation of the gesture changed it.
- Parameters
gesture (Gesture) – the gesture that deactivated
- get_all_used_gestures() Set[str]
- get_event_trigger_types() Set[str]
- get_bodypart_types() Set[str]
Module contents
stuff text