C3S Tracker: Issueshttps://redmine.c3s.cc/https://redmine.c3s.cc/favicon.ico2023-02-27T19:11:12ZC3S Tracker
Redmine collecting_society - Datenbank #1140 (Neu): Finish allocation processeshttps://redmine.c3s.cc/issues/11402023-02-27T19:11:12ZAlexander Blum
<a name="Background"></a>
<h1 >Background<a href="#Background" class="wiki-anchor">¶</a></h1>
<ul>
<li><code>Invoice.amount/shared_amount</code> are only helper attributes for testing the invoicing, should be replaced by
<code>UtilisationIndicators.invoice_amount</code> = <code>UtilizationIndicators.administration_fee</code> + <code>UtilizationIndicators.distribution_amount</code></li>
<li>relationship see also <a href="https://redmine.c3s.cc/projects/collecting_society/wiki/Databasemodels#Invoicing" class="external">database diagram</a></li>
<li><p>use <code>erpserver> db-console</code> for a tryton console to test the functions; maybe write a script with some objects initialized and import it there</p>
<pre>>>> Tariff = pool.get('tariff_system.tariff')
>>> tariff = Tariff(1)
</pre></li>
</ul>
<a name="Procedure"></a>
<h1 >Procedure<a href="#Procedure" class="wiki-anchor">¶</a></h1>
<ol>
<li>Write a <a href="https://github.com/C3S/collecting_society/blob/development/collecting_society.py#L829" class="external">wizard</a> to trigger the allocation process
<ul>
<li>can be triggered by
<ul>
<li>cron job</li>
<li>manual <a href="https://github.com/C3S/collecting_society/blob/development/collecting_society.xml#L800" class="external">action</a>
<ul>
<li>in Declaration Menu (action in entry view or on <a href="https://github.com/C3S/collecting_society/blob/development/collecting_society.py#L3973" class="external">selected</a> items in list view)</li>
<li>in Utilization Menu (action in entry view or on <a href="https://github.com/C3S/collecting_society/blob/development/collecting_society.py#L3973" class="external">selected</a> items in list view). Maybe we could skip that, but probable it's a valid usecase, like:
<ul>
<li>if some utilization (tariff bound!) of a declaration should be payed for and some other has issues to be resolved first</li>
<li>or a multi day festival event and the utilisations should be split.</li>
</ul></li>
</ul></li>
</ul></li>
<li>creates allocations with all corresponding utilisations connected, one allocation for each licensee</li>
<li>loops over different sets of declarations depending on where/how the action was triggered (algorithm should be generic, see next point confirm screen how this is done; not sure, how to handle the same action with different selections though, all other things should be similiar to the existing example):
<ul>
<li>declarations list with no selection: all suitable declarations with suitable utilisations (default for the cron job e.g. each day)</li>
<li>declarations list with selection: all suitable declaration.utilisations for each selected suitable declaration</li>
<li>declaration entry: all suitable declaration.utilisations for this declaration (if suitable)</li>
<li>utilisations list with no selection: all suitable selected utilisations</li>
<li>utilisations list with selection: all suitable utilisations in list</li>
<li>utilisation entry: this utilisation (if suitable)</li>
</ul></li>
<li>confirm screen (like in <a href="https://github.com/C3S/collecting_society/blob/development/collecting_society.py#L4194" class="external">fingerprint_merge</a>, maybe a tree list with declarations -> utilisations)
<ul>
<li>list of definitely selected utilisations/declarations</li>
<li>info on deselection of non suitable utilisations/declarations</li>
</ul></li>
</ul></li>
<li>Write a dataset <a href="https://github.com/C3S/collecting_society_docker/blob/development/volumes/shared/data/datasets
/device_message_fingerprint_merge.py#L29" class="external">triggering</a> the wizard for some Declarations
<ul>
<li>ensure, that non complete utilisation datasets for manual tests are still available</li>
</ul></li>
<li><p>Implement the Tariff and TariffSystem <code>formula</code>s from <a href="https://redmine.c3s.cc/projects/collecting_society/wiki/Databasemodels#Invoicing" class="external">database diagram</a></p>
<ul>
<li>fomulas should have the exact same plain value parameters as stated in the tariff system (check the latest version), so that parametrized calling is possible via <code>formula = getattr(tariff, 'method_name')</code>. they should <strong>not</strong> receive some tryton object, so they can be used (and tested) standalone</li>
<li><p>each formula should be one <strong>static</strong> method</p>
<ul>
<li><p>TariffSystem parameters: tariff system version (dot replaced with underscore); like</p>
<pre>@staticmethod
def formula_<tariffsystemversion>(...):
[...]
return invoice_amount
</pre></li>
<li><p>Tariff parameters: tariff code (includes version already, dot replaced with underscore); one for each tariff; like</p>
<pre>@staticmethod
def formula_<tariffcode>(...):
[...]
return base, relevance, adjustments
</pre></li>
<li><p>maybe better change the tariff_system.version.code, that it is save to use as function names in general and omit the replacements here</p></li>
</ul></li>
<li><p>add an <strong>instance</strong> method, to be able to easily fetch the corresponding formula from an instance</p>
<ul>
<li><p>TariffSystem</p>
<pre>def get_formula(self):
return getattr(self, f"formula_{self.version.replace(',', '_')}")
</pre></li>
<li><p>Tariff </p>
<pre>def get_formula(self):
return getattr(self, f"formula_{self.code.replace(',', '_')}")
</pre></li>
</ul></li>
</ul></li>
<li><p>Implement the <strong>instance</strong> methods</p>
<ul>
<li>params:
<ul>
<li>sample: indicator set
<ul>
<li>e.g. 'estimated', 'confirmed', might depend on tariff</li>
<li>maybe also 'all' to calculate all indicator sets, but then the return value has to be e.g. a dict with sample name as key and the tuple <code>(base, relevance, adjustments)</code> as value</li>
</ul></li>
<li>update: write or just calculate the results</li>
</ul></li>
<li>maybe rename <code>calculate_invoice_amount</code>, as all values are returned, or maybe better split invoice amount calculation (only invoice_amount returned) from calculating the distribution_amount and administration_fee (but not sure if this is needed by some other process)</li>
<li><p><code>Allocation.calculate_invoice_amount(self, sample, update=False)</code></p>
<pre>invoice_amount_sum = 0
distribution_amount_sum = 0
administration_fee_sum = 0
for utilisation self.utilisations:
invoice_amount, distribution_amount, administration_fee = \
utilisation.calculate_invoice_amount(sample, update)
invoice_amount_sum += invoice_amount
distribution_amount_sum += distribution_amount
administration_fee_sum += administration_fee
return invoice_amount, distribution_amount, administration_fee
</pre></li>
<li><p><code>Utilisation.calculate_invoice_amount(self, sample, update=False)</code></p>
<pre>utilisation = self
invoice_amount = self.tariff.calculate_invoice_amount(utilisation, sample, update)
return invoice_amount, distribution_amount, admnistration_fee
</pre></li>
<li><p><code>Tariff.calculate_invoice_amount(self, utilisation, sample, update=False)</code></p>
<pre>formula = self.get_formula()
context_indicators = getattr(utilisation.context, f'{sample}_indicators')
# generic way to get all and only indicators needed for all different formulas
import inspect # place import on top of file
formula_indicators = {
field: getattr(context_indicators, field)
for field in inspect.signature(formula).parameters
}
base, relevance, adjustments = formula(**formula_indicators):
invoice_amount, distribution_amount, administration_fee = \
tariff_system.calculate_invoice_amount(base, relevance, adjustments)
if update:
utilisation_indicators = getattr(utilisation, f'{sample}_indicators')
utilisation_indicators.base = base
utilisation_indicators.relevance = relevance
utilisation_indicators.adjustments = adjustments
utilisation_indicators.invoice_amount = invoice_amount
utilisation_indicators.distribution_amount = distribution_amount
utilisation_indicators.administration_fee = administration_fee
utilisation_indicators.save()
return invoice_amount, distribution_amount, administration_fee
</pre></li>
<li><p><code>TariffSystem.calculate_invoice_amount(self, base, relevance, adjustments)</code></p>
<pre>formula = self.get_formula()
invoice_amount = formula(base, relevance, adjustments)
distribution_amount = invoice_amount * self.administration_share
administration_fee = invoice_amount - distribution_amount
# check if results add up
return invoice_amount, distribution_amount, administration_fee
</pre></li>
</ul></li>
<li><p>Implement the same for <code>calculate_utilisation_indicators()</code>, maybe reuse in <code>calculate_invoice_amount()</code></p>
<ul>
<li><code>Utilisation.calculate_utilisation_indicators(self, sample, update)</code></li>
<li><code>Tariff.calculate_utilisation_indicators(self, utilisation, sample, update)</code></li>
<li>best case: each <a href="https://redmine.c3s.cc/projects/collecting_society/wiki/Databasemodels#Invoicing" class="external">formula o node</a> with a custom method to use, and one method to integrate all of them (e.g. another <code>calculate_invoice_split()</code> function)</li>
</ul></li>
<li><p>Implement a wizard to recalculate invoice amounts / utilsation indicators (prevent, if invoice is already issued)</p></li>
<li><p>Check the wizard to write invoices (<a href="https://github.com/C3S/collecting_society/blob/development/collecting_society.py#L829" class="external">exists</a> already)</p></li>
<li><p>Write datasets to trigger the invoice action for the Allocations</p>
<ul>
<li>ensure, that some allocations are still left to invoice for manual tests</li>
</ul></li>
</ol>
collecting_society - Webfrontend #1105 (Neu): Upgrade 3rd party libshttps://redmine.c3s.cc/issues/11052021-04-16T19:32:31ZAlexander Blum
<ul>
<li>jQuery</li>
<li>Bootstrap</li>
<li>jQuery-Fileupload</li>
<li>Datatables</li>
</ul>
collecting_society - Konzept #1016 (Zurückgestellt): Replace incremented C3S IDs with UUIDs?https://redmine.c3s.cc/issues/10162020-11-06T17:16:16ZNico H
<p>Ist es clever, bei Artist, Files, Creations, Releases, etc. eine fortlaufende Nummer zu verwenden?</p>
<p>Dies könnte später ein Information leakage sein, wie viele Datensätze in der Datenbank sind.<br>
Oder falls eine Schwachstelle existiert, ließen sich so weitere IDs leicht raten. </p>
collecting_society - Webfrontend #1005 (Neu): If list view is empty, redirect to add viewhttps://redmine.c3s.cc/issues/10052020-09-24T18:13:06ZThomas Mielke
<p>instead of showing an empty list (e.g. locations) staight forward to the add viel (e.g. add location). </p>
collecting_society - Userfeedback #953 (Neu): Release: add track -- why is there a track input fi...https://redmine.c3s.cc/issues/9532019-12-28T19:37:22ZThomas Mielke
<p>... as tracks can be selected from a list.</p>
<p>function of the field needs to be obvous. (previously we named it working title -- as the title of the creation).</p>
collecting_society - Schnittstellen #952 (Neu): implement works apihttps://redmine.c3s.cc/issues/9522019-12-19T22:29:09ZThomas Mielke
<p>see pad <a href="https://gpad.c3s.cc/p/bm3s/2019-01-12_api-specs">https://gpad.c3s.cc/p/bm3s/2019-01-12_api-specs</a></p>
collecting_society - Schnittstellen #951 (Neu): EchoPrint delete apihttps://redmine.c3s.cc/issues/9512019-12-19T21:42:59ZThomas Mielke
<p>we need an api to delete fingerprints from the EchoPrint server.</p>
<p>Currently the solr delete_query throws an exception, finding an illegal character 0xc8.</p>
<p>Tried to debug, but ptvsd has a bug too, preventing me to step through non-main threads. In ptvsd 5.0 the problem is solved, so I will wait for the release.</p>
collecting_society - Tests #950 (Neu): worker integration test can't delete web_userhttps://redmine.c3s.cc/issues/9502019-12-11T23:26:44ZThomas Mielke
<p>i.t.m. it is not possible to delete a web_user record rightaway. web_user ist too entangled with the tryton user, party, artist, etc. to have a record easyily deleted. the test currently generates random email addresses as a workaround for the email unique restraint.</p>
collecting_society - Konzept #945 (Neu): There exists a fixed set of contribuion roles wihtin DDE...https://redmine.c3s.cc/issues/9452019-12-01T19:11:45ZThomas Mielkecollecting_society - Webfrontend #937 (Zurückgestellt): JSON Web Tokenshttps://redmine.c3s.cc/issues/9372019-12-01T18:31:34ZAlexander Blum
<ul>
<li><a href="https://blog.logrocket.com/how-to-secure-a-rest-api-using-jwt-7efd83e71432/" class="external">JWT</a> (JSON Web Tokens)</li>
<li><a href="https://pyjwt.readthedocs.io/en/latest/" class="external">PyJWT</a> (Python implementation)</li>
<li>Found via <a href="https://www.privacyidea.org/about/features/" class="external">privacyIDEA</a>, a 2 Factor Authentication system, which is using JWT to communicate with other services using the 2FA feature of privacyIDEA.</li>
</ul>
collecting_society - Konzept #873 (Feedback): Fixed attributes of artists?https://redmine.c3s.cc/issues/8732019-03-20T10:33:11ZAlexander Blum
<ul>
<li>Should an artist be renameable or is it a new artist per definition?
<ul>
<li>References to this artist were commited by others, they should at least receive some notification or better something to actively acknowledge the renaming.</li>
</ul></li>
<li>Would it be ok, to have the group (yes/no) attribute of artists readonly after creation?
<ul>
<li>Changing this would have several consequences, which we could just circumvent by having this attribute read only.</li>
</ul></li>
</ul>
collecting_society - Datenbank #872 (Neu): Duplicate on deletionhttps://redmine.c3s.cc/issues/8722019-03-20T10:24:02ZAlexander Blum
<p>If an uncommited object is deleted before recommit, a duplicate needs to be created and all references should point to the duplicate instead.</p>
collecting_society - Webfrontend #871 (Neu): Add other track titles in add listshttps://redmine.c3s.cc/issues/8712019-03-20T10:20:33ZAlexander Blum
<p>Other track titles should be listed among the creation title (~"also known as").</p>
collecting_society - Konzept #867 (Neu): Autoclaim based on email?https://redmine.c3s.cc/issues/8672019-03-17T09:46:26ZAlexander Blum
<p>Do we want to have objects autoclaimed, if the email of an artist matches the email of the newly registered web user?</p>
<p>I suggest to not autoclaim, but to display those objects in a separate section of the claim interface.<br>
Maybe we should extend the <a href="https://redmine.c3s.cc/projects/repertoire/wiki/Workflows#Claim" class="external">workflow</a> by an "offered" state.</p>
collecting_society - Webfrontend #865 (Neu): Auto-redirect after successful uploadhttps://redmine.c3s.cc/issues/8652019-03-16T23:24:52ZThomas Mielke
<p>... also display processing stage on files overview.</p>