Administration #609

Limit resources of processing

Added by Alexander Blum over 5 years ago. Updated 3 months ago.

Status:ZurückgestelltStart date:
Priority:NormalDue date:
Assignee:Alexander Blum% Done:

50%

Category:-Estimated time:2.00 h
Target version:Repertoire 2) Testing phase II

Description

If it is possible to upgrade docker-compose version to 3, we might limit the resources with docker-compose statements:

Alternative (For further details, see Architecture):

Associated revisions

Revision 217a4bac
Added by Alexander Blum about 5 years ago

fixes #609: limits resources for processing service

History

#1 Updated by Alexander Blum over 5 years ago

  • Tracker changed from Unterstützung to Administration
  • Subject changed from Create missing VMs to Limit resources of processing
  • Description updated (diff)
  • Assignee set to Thomas Mielke
  • Target version set to 1) Testing phase I
  • Estimated time set to 10.00

#2 Updated by Alexander Blum about 5 years ago

  • Status changed from Neu to Erledigt
  • % Done changed from 0 to 100

Applied in changeset commit:c3s_ado_repertoire|217a4bac944a2d65e74f6cb9f1b5cb0324f83667.

#3 Updated by Alexander Blum about 5 years ago

  • Status changed from Erledigt to In Bearbeitung
  • Assignee changed from Thomas Mielke to Christoph Scheid

memory limit is not supported on production, maybe due to inactive cgroups https://github.com/moby/moby/issues/18420#issuecomment-212426212
is it possible, to activate cgroups?

#4 Updated by Alexander Blum about 5 years ago

  • Status changed from In Bearbeitung to Erledigt

Limiting of resources still doesn't work, although docker seems to detect the capability of the kernel now (new kernel installed, certain kernel parameters given).

But with 2 cores and 2G RAM instad of 1G there are no problems with web requests during file processing anymore.

As both CPU and RAM were not critical at any point before, the bottleneck still is not clear.

This ticket is closed, until further tests fail again.

#5 Updated by Christoph Scheid about 5 years ago

  • Status changed from Erledigt to Zurückgestellt
  • % Done changed from 100 to 50

Alexander Blum wrote:

Limiting of resources still doesn't work, although docker seems to detect the capability of the kernel now (new kernel installed, certain kernel parameters given).

But with 2 cores and 2G RAM instad of 1G there are no problems with web requests during file processing anymore.

so it works for you, now...

I wonder what stress testing will reveal...

btw do we have a ticket for stress testing?

As both CPU and RAM were not critical at any point before, the bottleneck still is not clear.

+1

This ticket is closed, until further tests fail again.

not closing, just deferring to later.

#6 Updated by Alexander Blum almost 5 years ago

  • Target version changed from 1) Testing phase I to 2) Testing phase II

#7 Updated by Alexander Blum over 4 years ago

  • Estimated time changed from 10.00 to 2.00

#8 Updated by Alexander Blum almost 4 years ago

  • Target version changed from 2) Testing phase II to Repertoire 2) Testing phase II

#9 Updated by Alexander Blum almost 4 years ago

  • Project changed from repertoire to collecting_society

#10 Updated by Christoph Scheid 3 months ago

Erledigt oder noch spannend?

¯_(ツ)_/¯

#11 Updated by Christoph Scheid 3 months ago

  • Assignee changed from Christoph Scheid to Alexander Blum

?

#12 Updated by Alexander Blum 3 months ago

it's still open. migration to the new compose version (newer than v3) is not done yet. in v1 it was possible, in v2 not, i think it's possible again with the new (versionless) format. it's just a reminder ticket, to try that out, whenever we finally migrate. migration is not trivial, as our setup is a bit more sophisticated, but the new version finally knows "tags" for services, so the split into compose files for each stage, testing and documentation could be reduced to a single compose.yml

#13 Updated by Alexander Blum 3 months ago

the task priority is coupled to the staging phase of the software. for development, it does not matter, but on staging/production - when many files are processed - the worker machine must not use all the resources

Also available in: Atom PDF