Commit cc30e00e authored by Erik Bjäreholt's avatar Erik Bjäreholt
Browse files

major improvements all over the docs

parent 55c3883f
......@@ -3,6 +3,13 @@ Architecture
Here we hope to clarify the architecture of ActivityWatch for you. Please file an issue or pull request if you think something is missing.
Dependency graph
----------------
The below is a graph of the fundamental dependencies between projects, these do not reflect the folder structure.
.. graphviz:: dependency.dot
Server
------
......@@ -10,14 +17,14 @@ Known as aw-server, it handles storage and retrieval of all activities/entries i
The server also hosts the Web UI (aw-webui) which does all communication with the server using the REST API.
Watchers
--------
Clients (watchers, importers, and observers)
--------------------------------------------
Since aw-server doesn't do any data collection on it's own, we need watchers that observe the world and sent the data off to aw-server for storage.
These utilize the :doc:`aw-client` library for making requests to the aw-server.
For a list of watchers, see :doc:`watchers`.
For a list of watchers, see :doc:`watchers`. For a list of importers see :doc:`importers`.
User interfaces
......@@ -25,13 +32,13 @@ User interfaces
ActivityWatch currently has two user interfaces, aw-qt and aw-webui.
- `aw-qt <https://github.com/ActivityWatch/aw-qt>`_ - Manages the server and watchers to make ActivityWatch easy to use for end-users.
- `aw-webui <https://github.com/ActivityWatch/aw-webui>`_ - Offers visualization and an overview of the database. Hosted by aw-server in the bundle.
- :gh-aw:`aw-qt` - Manages the server and watchers to make ActivityWatch easy to use for end-users.
- :gh-aw:`aw-webui` - Offers visualization and an overview of the database. Hosted by aw-server in the bundle.
Libraries
---------
Some of the logic of ActivityWatch is shared across the server and clients, for these cases we moved some logic into seperate libraries.
Some of the logic of ActivityWatch is shared across the server and clients, for these cases we moved some logic into separate libraries.
aw-core
^^^^^^^
......
Buckets and Events
==================
Data model
==========
Buckets
-----------
-------
The fundamental datacontainer in ActivityWatch, a bucket contains events and common metadata for those events (such as which type of events they are, where they were collected, and by what).
It is recommended to have one bucket per watcher and host. A bucket should always receive data from the same source.
For example, if we want to write a watcher that should track the currently active window we would first have it create a bucket named 'example-watcher-window_myhostname' and then start reporting events to that bucket (using heartbeats).
.. code-block:: javascript
bucket = {
"id": "aw-watcher-mywatcher_myhostname",
"id": "aw-watcher-test_myhostname",
"created": "2017-05-16T13:37:00.000000",
"name": "A short but descriptive human readable bucketname",
"type": "mybuckettype",
"client": "aw-watcher-mywatcher",
"hostname": "myhostname",
"type": "com.example.test", // Type of events in bucket
"client": "example-watcher-test", // Identifier of client software used to report data
"hostname": "myhostname", // Hostname of device where data was collected
}
In ActivityWatch we try and separate each kind of datapoint.
Normally what is most convenient is to have one bucket per client and host.
For example if we have a watcher which tracks info about the focused window on your computer we would upon startup create a bucket named 'aw-watcher-window_my-host-name' to fill all of our events with datapoints to.
Each bucket also has a buckettype which specifies the format of the events inside the buckets. See examples at :ref:`buckettypes`
For information about the "type" field, see examples at :ref:`event types`.
Events
-----------
------
The ActivityWatch event model is pretty simple, here's its representation in JSON:
The event model used by ActivityWatch is pretty simple, here is the JSON representation:
.. code-block:: javascript
event = {
"timestamp": "2016-04-27T15:23:55Z", // ISO8601 formatted timestamp
"duration": 3.14, // Duration in seconds
"data": {"key": "value"}, // A JSON object, the schema of this depends on the bucket type
"data": {"key": "value"}, // A JSON object, the schema of this depends on the event type
}
It should be noted that all timestamps are stored as UTC. Timezone information (UTC offset) is currently discarded.
The content in the "data" field could be any JSON object, but it is recommended that every event in a bucket should follow some format depending on the buckettype so the data is easy to analyze.
Event types
```````````
Buckettypes
-----------
To separate different types of data in ActivityWatch there is the event type. A buckets event type specified the schema of the events in the bucket.
The buckettype specifies the format of the data in the event, so for example for the bucket of our focused window watcher we could have the buckettype 'currentwindow'.
The buckettype is just a ordinary string, but it is good for clients who want to visualize the data in ActivityWatch to know what the buckets contain.
By creating standards for watchers to use we enable easier transformation and visualization.
web.tab.current
~~~~~~~~~~~~~~~
An event type for the currently active webbrowser tab.
.. code-block:: javascript
......@@ -54,26 +58,28 @@ web.tab.current
url: string,
title: string,
audible: bool,
icognito: bool,
incognito: bool,
}
app.editor.activity
~~~~~~~~~~~~~~~~~~~
An event type for tracking the currently edited file.
.. code-block:: javascript
{
file: string, (full path to file)
project: string, (full path of cwd)
language: string, (name of language of the file)
file: string, // full path to file
project: string, // full path of cwd
language: string, // name of language of the file
}
currentwindow
~~~~~~~~~~~~~
.. note::
There are suggestions to improve/change this format
(see `GitHub issue
<https://github.com/ActivityWatch/activitywatch/issues/201>`_)
(see :issue:`201`)
.. code-block:: javascript
......@@ -82,16 +88,15 @@ currentwindow
title: string,
}
afkstatus
~~~~~~~~~
.. note::
There are suggestions to improve/change this format
(see `GitHub issue
<https://github.com/ActivityWatch/activitywatch/issues/201>`_)
(see :issue:`201`)
.. code-block:: javascript
{
status: string ("afk" or "not-afk")
status: string // "afk" or "not-afk"
}
......@@ -24,24 +24,24 @@ Web UI:
Watchers:
- Improved stability of client event queues (`see this PR <https://github.com/ActivityWatch/aw-client/pull/28>`_)
- Improved stability of client event queues (:gh-aw:`see this PR <aw-client/pull/28>`)
Other:
- Windows: Console window and taskbar icon now hidden by default (`issue #139 <https://github.com/ActivityWatch/activitywatch/issues/139>`_)
- All issues assigned to the v0.8 milestone can be found `on GitHub <https://github.com/ActivityWatch/activitywatch/milestone/1>`_
- Windows: Console window and taskbar icon now hidden by default (:issue:`139`)
- All issues assigned to the v0.8 milestone can be found :gh-aw:`on GitHub <activitywatch/milestone/1>`
v0.7.1
--------
- Actually fixed the timezone issue in the web UI (`issue #117 <https://github.com/ActivityWatch/activitywatch/issues/117>`_).
- All issues assigned to the v0.7 milestone can be found `on GitHub <https://github.com/ActivityWatch/activitywatch/milestone/4>`_.
- Actually fixed the timezone issue in the web UI (:issue:`117`).
- All issues assigned to the v0.7 milestone can be found :gh-aw:`on GitHub <activitywatch/milestone/4>`.
v0.7.0b4
--------
- The ActivityWatch WebExtension is now supported from this version forward, see the announcement `on the forum <https://forum.activitywatch.net/t/you-can-now-track-your-web-browsing-with-activitywatch/28>`_.
- (Not really, see v0.7.1) Fixed pesky timezone issue in web UI (`issue #117 <https://github.com/ActivityWatch/activitywatch/issues/117>`_).
- The ActivityWatch WebExtension is officially supported from this version forward, see the announcement `on the forum <https://forum.activitywatch.net/t/you-can-now-track-your-web-browsing-with-activitywatch/28>`_.
- (Not really, see v0.7.1) Fixed pesky timezone issue in web UI (:issue:`117`).
- Fixed bug on macOS where keyboard activity would not be used to detect AFK state.
- Fixed packaging bugs (macOS, PyInstaller).
- The web extension now has a better look and notifies if connection to server failed.
......
......@@ -39,8 +39,16 @@ extensions = [
'sphinx.ext.viewcode',
'sphinx.ext.todo',
'sphinx.ext.graphviz',
'sphinx.ext.extlinks',
]
extlinks = {
'issue': ('https://github.com/Activity/issues/%s', 'issue #'),
'gh': ('https://github.com/%s', ''),
'gh-user': ('https://github.com/%s', '@'),
'gh-aw': ('https://github.com/ActivityWatch/%s', ''),
}
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
......
......@@ -6,11 +6,6 @@ Development
We recommend you follow Kenneth Reitz folder structure guide when writing Python programs which will be under the control of the ActivityWatch organisation: http://docs.python-guide.org/en/latest/writing/structure/
Dependency graph
----------------
.. graphviz:: dependency.dot
Working with submodules
-----------------------
......
......@@ -22,6 +22,6 @@ Fetching Data
If you want to fetch data from aw-server for visualization, exporting, backup or something we have not yet thought of, there are a few ways you can do this:
* `Exporting a Bucket <features/exporting-data.html>` If you want a complete dump of all events of bucket
* `Bucket REST API <./rest.html#get-events>`_ If you want to export raw events in a specific time interval from a bucket
* `Writing a Query <./querying-data.html#writing-a-query>`_ If you want to summarize/aggregate one or more buckets into more easily readable data
* `Exporting a Bucket <features/exporting-data>`_ If you want a complete dump of all events of bucket
* `Bucket REST API <get-events>`_ If you want to export raw events in a specific time interval from a bucket
* `Writing a Query <writing-a-query>`_ If you want to summarize/aggregate one or more buckets into more easily readable data
<!-- You aren't able to record what you think, but you can approximate by asking: What could be on your mind? -->
You aren't able to record what you think, but with ActivityWatch you can accurately answer the question of what has been on your mind.
How much you might be thinking about something, says a lot about how you spend your time/attention/thoughts.
## Backstory
It was 2013, I was just about to start university and had been soaked in transhumanist ideas and hacker culture for much of my time in high school. I had been reading a bit about brain computer interfaces, and in a moment of clarity I realized its potentially enormous impact. I wrote a private note to myself about my realization and that I should definitely join in on making it a reality when the right time comes.
During the same time, I was obsessively logging what I did using: automated time-trackers (like ActivityWatch), a great spreadsheet, a diary, the GPS in my phone (RIP battery), a step/sleep-tracker (Fitbit at the time), version control, etc.
## Now
Now we're building it
## Future
#### Building new types of privacy-aware services which require data collection
If a lot is collected by the user, new applications could be developed that utilize that data.
See for example [Thankful]. You might also consider the possibility of a [aggregated newsfeed with an open-source recommendation engine].
#### Ubiquitous recording for meaningful information about the past
> "We live in an interesting time when more and more of our actions can be in some way recorded and played back without our intervention. \[...] There's voice recording technology. Web browsing history. Live desktop video recording and playback. Heck, some folks \[...] have shown us a taste of the future as power-users of autonomous or assisted self-recording technologies. Go-pro and other consumer tech products are thriving as they discover / cherry-pick / surface compelling use cases. I haven't experienced general-purpose AI which is quite up to the job of organizing my notes for me. But we're close to having ubiquitous recording (and storing bits is the important part -- facebook didn't start with entity tags on day 1 but has been able to retroactively infer and index these). There's no way to record everything with perfect fidelity, because that would require us to preserve as many bits as there are in reality (which violates physical constraints) but there's a lot we can do to improve. There are still unexplored frontiers, like recording, transmitting, and playing back one's thoughts (which I don't think we should consider science fiction, just somewhat expensive and contentious to make viable). Suffice to say, interfaces (there aren't great memex-like ways to create graph based notes with semantic, taggable entities), politics and logistics of services competing to silo our information, and insufficient AI to infer our meaning and, in fact, to de-duplicate our thoughts and those of others (read: https://distill.pub/2017/research-debt) are major barriers which conspire against making mind-mapping and organizing one's life's work frictionless."
> Michael Karpeles in a [comment on Facebook](https://www.facebook.com/michael.karpeles/posts/10103225650726950?comment_id=10103225680237810&comment_tracking=%7B%22tn%22%3A%22R1%22%7D)
History
=======
This section is dedicated to the past, present, and future of ActivityWatch.
.. We aren't (yet) able to record what we think, but we can approximate by asking: What did we pay attention to?
We aren't (yet) able to automatically record our own thoughts, but with ActivityWatch we can record what were thinking about, What projects we worked on, which ideas we read about, the culture we enjoyed, and who created that culture.
We've thought a lot about what ActivityWatch could be, and what that could lead to. We unfortunately haven't made as great a job of writing things down as we should have but here's an attempt.
We believe that studying our own behavior can also help us identify and get rid of bad habits.
We believe that open source tools like ActivityWatch can be used to crowdsource an open set of research data, enabling entirely new research possibilities regarding the effect of different activities on human psychology and behavior.
Past
----
It was 2013, I was just about to start university and had been soaked in transhumanist ideas and hacker culture for most of my adolescent life. I had been reading a bit about brain computer interfaces, and in a moment of clarity I realized its potentially enormous impact. I wrote a private note to myself about my realization and that I should definitely join in on making it a reality when the right time comes.
During the same time, I was obsessively logging what I did using: automated time-trackers (like ActivityWatch), a great spreadsheet, a diary, the GPS in my phone (RIP battery), a step/sleep-tracker (Fitbit at the time), version control, etc.
:gh-user:`ErikBjare` started building a prototype on Dec 30, 2014. In April 2016 he started working on a rewrite (that included the client-server model) that then became the foundation for ActivityWatch is today. :gh-user:`johan-bjareholt` became a regular contributor some time later in 2016, and has since been the second largest contributor to the project by a wide margin.
Now
---
We're building it, and we've only just begun.
Future
------
Building new types of privacy-aware services which require data collection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If a lot is collected by the user, new applications could be developed that utilize that data.
See for example :gh:`Thankful <SuperuserLabs/thankful>`. Or my proposal for a `self-hosted aggregated newsfeed, with a highly customizable recommendation engine <https://erik.bjareholt.com/wiki/importance-of-open-recommendation-systems/>`_.
Ubiquitous recording for meaningful information about the past
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"We live in an interesting time when more and more of our actions can be in some way recorded and played back without our intervention. \[...] There's voice recording technology. Web browsing history. Live desktop video recording and playback. Heck, some folks \[...] have shown us a taste of the future as power-users of autonomous or assisted self-recording technologies. Go-pro and other consumer tech products are thriving as they discover / cherry-pick / surface compelling use cases. I haven't experienced general-purpose AI which is quite up to the job of organizing my notes for me. But we're close to having ubiquitous recording (and storing bits is the important part -- facebook didn't start with entity tags on day 1 but has been able to retroactively infer and index these). There's no way to record everything with perfect fidelity, because that would require us to preserve as many bits as there are in reality (which violates physical constraints) but there's a lot we can do to improve. There are still unexplored frontiers, like recording, transmitting, and playing back one's thoughts (which I don't think we should consider science fiction, just somewhat expensive and contentious to make viable). Suffice to say, interfaces (there aren't great memex-like ways to create graph based notes with semantic, taggable entities), politics and logistics of services competing to silo our information, and insufficient AI to infer our meaning and, in fact, to de-duplicate our thoughts and those of others (read: https://distill.pub/2017/research-debt) are major barriers which conspire against making mind-mapping and organizing one's life's work frictionless."
:gh-user:`mekarpeles` in a `comment on Facebook <https://www.facebook.com/michael.karpeles/posts/10103225650726950?comment_id=10103225680237810>`_.
Importers
=========
ActivityWatch can't track everything, so sometimes you might want to import data into ActivityWatch from another source.
There aren't many yet, but here are some attempts:
- :gh-aw:`aw-importer-smartertime`, imports from `smartertime`_ (Android time tracker).
- LastFM importer, :gh-user:`ErikBjare` has code for it somewhere, ask him if you're interested.
.. _smartertime: https://play.google.com/store/apps/details?id=com.smartertime&hl=en
......@@ -15,9 +15,11 @@ Welcome to the ActivityWatch documentation!
introduction
getting-started
faq
features
faq
history
watchers
importers
.. _dev-docs:
......@@ -25,14 +27,14 @@ Welcome to the ActivityWatch documentation!
:maxdepth: 3
:caption: Developer documentation
extending
development
architecture
buckets-and-events
api-reference
development
extending
writing-watchers
querying-data
installing-from-source
api-reference
rest
changelog
......
1. Install Git for Windows (including Git Bash)
1. Install MinGW
1. Rename C:/MinGW/mingw-make.exe to C:/MinGW/make.exe
1. `cp C:\\MinGW\\mingw32-make.exe C:\\MinGW\\make.exe`
1. Set PATH to use MinGW
1. `SET PATH=C:\\MinGW\\bin;%PATH%`
1. Install Python 3.5.4
1. Install PyInstaller
1. `pip install pyinstaller`
1. Add PyInstaller script to PATH: `SET PATH=C:\\Users\User\\AppData\\Roaming\\Python\\Python35\\Scripts`
Installing from source (on Windows)
===================================
This was a guide hastily written by :gh-user:`ErikBjare` when he had to build on Windows once, it is not complete.
- Install Git for Windows (including Git Bash)
- Install MinGW
- Rename C:/MinGW/mingw-make.exe to C:/MinGW/make.exe
- :code:`cp C:\\MinGW\\mingw32-make.exe C:\\MinGW\\make.exe`
- Set PATH to use MinGW
- :code:`SET PATH=C:\\MinGW\\bin;%PATH%`
- Install Python 3.5.4
- Install PyInstaller
- :code:`pip install pyinstaller`
- Add PyInstaller script to PATH: :code:`SET PATH=C:\\Users\User\\AppData\\Roaming\\Python\\Python35\\Scripts`
......@@ -4,7 +4,7 @@ Installing from source
Here's the guide to installing ActivityWatch from source. If you are just looking to try it out, see the getting started guide instead.
.. note::
This is written for Linux and macOS. For Windows the build process is more complicated and we therefore suggest using the pre-built packages instead on that operating system.
This is written for Linux and macOS. For Windows the build process is more complicated and we therefore suggest using the pre-built packages instead on that operating system (but if you really have to, see :doc:`this guide <installing-from-source-on-windows>`).
Cloning the submodules
----------------------
......@@ -57,7 +57,7 @@ Now activate the virtualenv in your current shell session:
source ./venv/Scripts/activate
# For fish users:
source ./venv/bin/activate.fish
Building and installing
......@@ -109,19 +109,18 @@ Please report all issues you might have so we can make things easier for future
Packaging your changes
----------------------
If you made some changes and want to run your code as you would normally do outside of the development environment you will need to install pyinstaller and python3-dev and package activitywatch
If you made some changes and want to create a proper build with portable executables (like normal ActivityWatch releases) you need to install :code:`pyinstaller` (and on Debian-like distros :code:`python3-dev`).
.. code-block:: sh
pip3 install pyinstaller
apt install python3-dev
Then package it
apt install python3-dev # Or equivalent for your Linux distribution
pip3 install --user pyinstaller
Then simply run the following to package it:
.. code-block:: sh
make package
When the packaging is done you will have :code:`./dist` folder where you can find a zipped version and an unzipped :code:`activitywatch` folder, you can move or copy that folder anywhere you need and set :code:aw-qt to run from startup.
When the packaging is done you will have :code:`./dist` folder where you can find a zipped version and an unzipped :code:`activitywatch` folder, you can move or copy that folder anywhere you need and set :code:`aw-qt` to run from startup.
......@@ -8,25 +8,37 @@ ActivityWatch comes bundled with two watchers by default:
- `aw-watcher-afk <https://github.com/ActivityWatch/aw-watcher-afk>`_ - Watches for mouse & keyboard activity to detect if the user is active.
- `aw-watcher-window <https://github.com/ActivityWatch/aw-watcher-window>`_ - Watches the active window and its title.
Extras
------
The default watchers are collecting some of the most important data.
But there is more to collect, so here are some other watchers that let you do so.
- `aw-watcher-web <https://github.com/ActivityWatch/aw-watcher-web>`_ - A browser extension that watches the active tab and its title along with its URL.
- `aw-watcher-vscode <https://github.com/ActivityWatch/aw-watcher-vscode>`_ - A Visual Studio Code extension that watches your editor activity
- `aw-watcher-vim <https://github.com/ActivityWatch/aw-watcher-vim>`_ - (Beta) A Vim extension that watches your editor activity
Browser watchers
----------------
Watches properties of the active tab like title, URL, and incognito state.
- :gh-aw:`aw-watcher-web` - The official browser extension, supports Chrome and Firefox.
Editor watchers
---------------
Watches the actively edited file and associated metadata like , project name (folder name of git root)
In development
- :gh-aw:`aw-watcher-vim` - vim extension, by :gh-user:`johan-bjareholt` and :gh-user:`ahnlabb`.
- :gh-aw:`aw-watcher-vscode` - Visual Studio Code extension, by :gh-user:`Otto-AA`.
- :gh:`pauldub/activity-watch-mode` - emacs mode forked from wakatime-mode, by :gh-user:`pauldub`.
Media watchers
--------------
- `aw-watcher-spotify <https://github.com/ActivityWatch/aw-watcher-spotify>`_ - (Beta) Uses the Spotify Web API to get the active track.
- `aw-watcher-chromecast <https://github.com/ActivityWatch/aw-watcher-chromecast>`_ - (WIP) Watches whatever your Chromecast is up to.
If you want to more accurately track media consumption.
- :gh-aw:`aw-watcher-spotify` - (Beta) Uses the Spotify Web API to get the active track.
- :gh-aw:`aw-watcher-chromecast` - (not working yet) Watches what is playing on you Chromecast device.
- :gh-aw:`aw-watcher-openvr` - (not working yet) Watches active VR applications.
Custom watchers
---------------
For help on how to write your own watcher, see `writing-watchers`.
Have you written one yourself? Send us a PR to have it included!
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment