Zammad: Badan artikel yang terlalu besar gagal mengimpor OTRS

Dibuat pada 1 Jul 2018  ·  9Komentar  ·  Sumber: zammad/zammad

Halo,

selain https://github.com/zammad/zammad/issues/1467 saya masih belum menyerah mengimpor data OTRS.
(Saya pikir tiket yang lebih tua bisa ditutup)

Tugas:

Dengan zammad 2.5 mencoba mengimpor data OTRS (~ 6000 tiket.

Hasil:

Coba 1:
Impor melalui WebGUI tidak berhasil, proses impor dihentikan setelah beberapa saat. Mungkinkah ada sesuatu seperti batas waktu?

Coba 2:
Impor melalui konsol lebih baik, tetapi impor gagal pada tiket dengan bodi besar (saya anggap ini adalah lampiran) dengan kesalahan berikut (lihat pengecualian di bawah). Menjalankan perintah berikut:

bundel rel exec c

Setting.set ('import_otrs_endpoint', 'http://123.456.789.012/otrs/public.pl?Action=ZammadMigrator')
Setting.set ('import_otrs_endpoint_key', 'IhAvEaVeRySeCrEtKeY')
Setting.set ('import_mode', true)
Setting.set ('system_init_done', false)
Impor :: OTRS.start

Pertanyaan:

  • Bisakah saya menaikkan batas badan di sumber untuk waktu proses impor atau akankah saya mendapat masalah setelahnya saat mengakses tiket?
  • Atau Apakah mungkin untuk melanjutkan mengimpor tiket yang tersisa
  • Saya menggunakan versi komposer-buruh pelabuhan Anda dari zammad. Di dokumen impor Anda menulis "Hentikan semua proses Zammad". Proses / wadah mana yang harus saya hentikan (terutama saat menggunakan versi buruh pelabuhan)

salam Hormat
MrDigit

Lacak untuk pengecualian:

W, [2018-07-01T12:58:20.356829 #76]  WARN -- : zammad-memcached:11211 failed (count: 0) RuntimeError: Already writing to socket
/opt/zammad/app/models/ticket/article.rb:301:in `check_body': body if article is to large, 3702347 chars - only 1500000 allowed (Exceptions::UnprocessableEntity)
        from /usr/local/bundle/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:413:in `block in make_lambda'
        from /usr/local/bundle/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:197:in `block (2 levels) in halting'
        from /usr/local/bundle/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:601:in `block (2 levels) in default_terminator'
        from /usr/local/bundle/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:600:in `catch'
        from /usr/local/bundle/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:600:in `block in default_terminator'
        from /usr/local/bundle/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:198:in `block in halting'
        from /usr/local/bundle/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:507:in `block in invoke_before'
        from /usr/local/bundle/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:507:in `each'
        from /usr/local/bundle/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:507:in `invoke_before'
        from /usr/local/bundle/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:130:in `run_callbacks'
        from /usr/local/bundle/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:827:in `_run_create_callbacks'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/callbacks.rb:340:in `_create_record'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/timestamp.rb:95:in `_create_record'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/persistence.rb:559:in `create_or_update'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/callbacks.rb:336:in `block in create_or_update'
        from /usr/local/bundle/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:131:in `run_callbacks'
        from /usr/local/bundle/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:827:in `_run_save_callbacks'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/callbacks.rb:336:in `create_or_update'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/persistence.rb:129:in `save'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/validations.rb:44:in `save'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/attribute_methods/dirty.rb:35:in `save'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/transactions.rb:308:in `block (2 levels) in save'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/transactions.rb:384:in `block in with_transaction_returning_status'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:235:in `block in transaction'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/connection_adapters/abstract/transaction.rb:194:in `block in within_new_transaction'
        from /usr/local/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/connection_adapters/abstract/transaction.rb:191:in `within_new_transaction'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:235:in `transaction'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/transactions.rb:210:in `transaction'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/transactions.rb:381:in `with_transaction_returning_status'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/transactions.rb:308:in `block in save'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/transactions.rb:323:in `rollback_active_record_state!'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/transactions.rb:307:in `save'
        from /usr/local/bundle/gems/activerecord-5.1.5/lib/active_record/suppressor.rb:42:in `save'
        from /opt/zammad/lib/import/otrs/article.rb:64:in `create'
        from /opt/zammad/lib/import/otrs/article.rb:49:in `create_or_update'
        from /opt/zammad/lib/import/otrs/article.rb:37:in `import'
        from /opt/zammad/lib/import/otrs/article.rb:31:in `initialize'
        from /opt/zammad/lib/import/base_factory.rb:35:in `new'
        from /opt/zammad/lib/import/base_factory.rb:35:in `create_instance'
        from /opt/zammad/lib/import/base_factory.rb:11:in `block in import_action'
        from /opt/zammad/lib/import/base_factory.rb:39:in `each'
        from /opt/zammad/lib/import/base_factory.rb:39:in `import_loop'
        from /opt/zammad/lib/import/base_factory.rb:9:in `import_action'
        from /opt/zammad/lib/import/otrs/ticket.rb:40:in `import'
        from /opt/zammad/lib/import/otrs/ticket.rb:30:in `initialize'
        from /opt/zammad/lib/import/base_factory.rb:35:in `new'
        from /opt/zammad/lib/import/base_factory.rb:35:in `create_instance'
        from /opt/zammad/lib/import/base_factory.rb:11:in `block in import_action'
        from /opt/zammad/lib/import/base_factory.rb:39:in `each'
        from /opt/zammad/lib/import/base_factory.rb:39:in `import_loop'
        from /opt/zammad/lib/import/base_factory.rb:9:in `import_action'
        from /opt/zammad/lib/import/otrs.rb:147:in `import_action'
        from /opt/zammad/lib/import/otrs.rb:137:in `imported?'
        from /opt/zammad/lib/import/otrs.rb:101:in `block (3 levels) in threaded_import'
        from /opt/zammad/lib/import/otrs.rb:95:in `loop'
        from /opt/zammad/lib/import/otrs.rb:95:in `block (2 levels) in threaded_import'
        from /usr/local/bundle/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
bug import verified
Sumber
picture MrDigit

Semua 9 komentar

Hai @MrDigit , DB mana yang akan digunakan? Saya pikir terbatas dari DB, lihat baris ini di log:

/opt/zammad/app/models/ticket/article.rb:301:in check_body': body if article is to large, 3702347 chars - only 1500000 allowed (Exceptions::UnprocessableEntity) from /usr/local/bundle/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:413:inblock in make_lambda'

NeverMin picture NeverMin pada 3 Jul 2018

Halo @Neverin ,

karena batas tersebut di-hardcode ke article.rb kesalahan ini independen dari backend DB yang dipilih.
( batas = 1_500_000 , lihat di bawah)

def check_body
    return true if body.blank?
    limit = 1_500_000
    current_length = body.length
    return true if body.length <= limit

    raise Exceptions::UnprocessableEntity, "body if article is to large, #{current_length} chars - only #{limit} allowed" if !ApplicationHandleInfo.postmaster?

    logger.warn "WARNING: cut string because of database length #{self.class}.body(#{limit} but is #{current_length})"
    self.body = body[0, limit]
end
MrDigit picture MrDigit pada 3 Jul 2018

Halo @MrDigit , Ya, Anda dapat memperbaiki (mis. 4_000_000) dan mencoba, tetapi, DB mana yang akan digunakan untuk Zammad? MySQL atau postgresql? Apa itu versi OTRS?

NeverMin picture NeverMin pada 4 Jul 2018

Halo @Neverin ,

sudah menaikkan batas ke beberapa nilai yang lebih tinggi, impor bekerja setelahnya. Pertanyaannya adalah mengapa limitnya disetel ke 1,5 MByte? Karena zammad akan meledak jika benda yang lebih besar disimpan atau ini hanya nilai sewenang-wenang?

Mengenai pertanyaan Anda: Versi docker-compose dari zammad yang saya gunakan (https://github.com/zammad/zammad-docker-compose) menggunakan postgresql, saya menyimpan lampiran di sistem file. OTRS adalah versi 5.

MrDigit picture MrDigit pada 4 Jul 2018

Halo @MrDigit , Pertanyaan bagus, mungkin izinkan saya bertanya ke @thorsteneckel .

JFI: https://github.com/zammad/zammad/blob/develop/db/migrate/20120101000010_create_ticket.rb#L175

DIEDIT:
Bisakah Anda memindahkan masalah ini ke Komunitas ?

NeverMin picture NeverMin pada 4 Jul 2018

Hai @rDigit

Pertanyaannya adalah mengapa limitnya disetel ke 1,5 MByte?

Saya pikir nilai ini sudah lama sekali dan mungkin sudah usang. Terima kasih telah menggali lebih dalam, menyelesaikan masalah Anda dan membagikannya di sini. Kami akan melihat lebih dekat saat menangani masalah ini.

thorsteneckel picture thorsteneckel pada 23 Jul 2018

Ini tidak terbatas pada impor OTRS saja.
Ini akan menyerang di beberapa titik.

Ketika mencoba mereproduksi ini, bahkan dengan 15000 karakter, saya akan tetap menerima kesalahan (catatan: Saya harus beralih ke API di sini karena UI kami tidak menyukainya):

F, [2019-07-03T16:36:15.330582 #18433-86302140] FATAL -- :
F, [2019-07-03T16:36:15.330620 #18433-86302140] FATAL -- : actionpack (5.1.7) lib/action_dispatch/http/parameters.rb:115:in `rescue in parse_formatted_parameters'
actionpack (5.1.7) lib/action_dispatch/http/parameters.rb:109:in `parse_formatted_parameters'
actionpack (5.1.7) lib/action_dispatch/http/request.rb:361:in `block in POST'
rack (2.0.7) lib/rack/request.rb:59:in `fetch'
rack (2.0.7) lib/rack/request.rb:59:in `fetch_header'
actionpack (5.1.7) lib/action_dispatch/http/request.rb:360:in `POST'
actionpack (5.1.7) lib/action_dispatch/http/parameters.rb:53:in `parameters'
actionpack (5.1.7) lib/action_dispatch/http/filter_parameters.rb:41:in `filtered_parameters'
actionpack (5.1.7) lib/action_controller/metal/instrumentation.rb:21:in `process_action'
actionpack (5.1.7) lib/action_controller/metal/params_wrapper.rb:252:in `process_action'
activerecord (5.1.7) lib/active_record/railties/controller_runtime.rb:22:in `process_action'
actionpack (5.1.7) lib/abstract_controller/base.rb:124:in `process'
actionview (5.1.7) lib/action_view/rendering.rb:30:in `process'
actionpack (5.1.7) lib/action_controller/metal.rb:189:in `dispatch'
actionpack (5.1.7) lib/action_controller/metal.rb:253:in `dispatch'
actionpack (5.1.7) lib/action_dispatch/routing/route_set.rb:49:in `dispatch'
actionpack (5.1.7) lib/action_dispatch/routing/route_set.rb:31:in `serve'
actionpack (5.1.7) lib/action_dispatch/journey/router.rb:50:in `block in serve'
actionpack (5.1.7) lib/action_dispatch/journey/router.rb:33:in `each'
actionpack (5.1.7) lib/action_dispatch/journey/router.rb:33:in `serve'
actionpack (5.1.7) lib/action_dispatch/routing/route_set.rb:844:in `call'
omniauth (1.9.0) lib/omniauth/strategy.rb:192:in `call!'
omniauth (1.9.0) lib/omniauth/strategy.rb:169:in `call'
omniauth (1.9.0) lib/omniauth/strategy.rb:192:in `call!'
omniauth (1.9.0) lib/omniauth/strategy.rb:169:in `call'
omniauth (1.9.0) lib/omniauth/strategy.rb:192:in `call!'
omniauth (1.9.0) lib/omniauth/strategy.rb:169:in `call'
omniauth (1.9.0) lib/omniauth/strategy.rb:192:in `call!'
omniauth (1.9.0) lib/omniauth/strategy.rb:169:in `call'
omniauth (1.9.0) lib/omniauth/strategy.rb:192:in `call!'
omniauth (1.9.0) lib/omniauth/strategy.rb:169:in `call'
omniauth (1.9.0) lib/omniauth/strategy.rb:192:in `call!'
omniauth (1.9.0) lib/omniauth/strategy.rb:169:in `call'
omniauth (1.9.0) lib/omniauth/strategy.rb:192:in `call!'
omniauth (1.9.0) lib/omniauth/strategy.rb:169:in `call'
omniauth (1.9.0) lib/omniauth/strategy.rb:192:in `call!'
omniauth (1.9.0) lib/omniauth/strategy.rb:169:in `call'
omniauth (1.9.0) lib/omniauth/strategy.rb:192:in `call!'
omniauth (1.9.0) lib/omniauth/strategy.rb:169:in `call'
omniauth (1.9.0) lib/omniauth/builder.rb:64:in `call'
rack (2.0.7) lib/rack/etag.rb:25:in `call'
rack (2.0.7) lib/rack/conditional_get.rb:38:in `call'
rack (2.0.7) lib/rack/head.rb:12:in `call'
rack (2.0.7) lib/rack/session/abstract/id.rb:232:in `context'
rack (2.0.7) lib/rack/session/abstract/id.rb:226:in `call'
actionpack (5.1.7) lib/action_dispatch/middleware/cookies.rb:613:in `call'
actionpack (5.1.7) lib/action_dispatch/middleware/callbacks.rb:26:in `block in call'
activesupport (5.1.7) lib/active_support/callbacks.rb:97:in `run_callbacks'
actionpack (5.1.7) lib/action_dispatch/middleware/callbacks.rb:24:in `call'
actionpack (5.1.7) lib/action_dispatch/middleware/debug_exceptions.rb:59:in `call'
actionpack (5.1.7) lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
railties (5.1.7) lib/rails/rack/logger.rb:36:in `call_app'
railties (5.1.7) lib/rails/rack/logger.rb:24:in `block in call'
activesupport (5.1.7) lib/active_support/tagged_logging.rb:69:in `block in tagged'
activesupport (5.1.7) lib/active_support/tagged_logging.rb:26:in `tagged'
activesupport (5.1.7) lib/active_support/tagged_logging.rb:69:in `tagged'
railties (5.1.7) lib/rails/rack/logger.rb:24:in `call'
actionpack (5.1.7) lib/action_dispatch/middleware/remote_ip.rb:79:in `call'
actionpack (5.1.7) lib/action_dispatch/middleware/request_id.rb:25:in `call'
rack (2.0.7) lib/rack/method_override.rb:22:in `call'
rack (2.0.7) lib/rack/runtime.rb:22:in `call'
activesupport (5.1.7) lib/active_support/cache/strategy/local_cache_middleware.rb:27:in `call'
actionpack (5.1.7) lib/action_dispatch/middleware/executor.rb:12:in `call'
actionpack (5.1.7) lib/action_dispatch/middleware/static.rb:125:in `call'
rack (2.0.7) lib/rack/sendfile.rb:111:in `call'
railties (5.1.7) lib/rails/engine.rb:522:in `call'
railties (5.1.7) lib/rails/railtie.rb:185:in `public_send'
railties (5.1.7) lib/rails/railtie.rb:185:in `method_missing'
puma (3.12.1) lib/puma/configuration.rb:227:in `call'
puma (3.12.1) lib/puma/server.rb:660:in `handle_request'
puma (3.12.1) lib/puma/server.rb:474:in `process_client'
puma (3.12.1) lib/puma/server.rb:334:in `block in run'
puma (3.12.1) lib/puma/thread_pool.rb:135:in `block in spawn_thread'
logging (2.2.2) lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

Respons API kosong.

Saya tidak bisa, pada titik mana pun, mendorong API ke batas di atas, karena Zammad tidak akan mengambil muatan saya.

Saya mencoba dengan 1500001 karakter (yang tidak terlalu disukai Browser saya) dan setelahnya dengan lebih sedikit 15000 karakter (nilai ini dapat ditempel di dalam Zammad). Tapi sekali lagi, jika artikelnya terlalu besar, itu hanya akan berhenti mengambil pasta.

1500001 Namun bukan 1,5MB, melainkan sekitar 1,14MB yang bahkan bukan batasnya. Saya dapat menempelkan sekitar 494320 karakter (teks biasa!), Setelah itu pengetikan manual masih berfungsi. Terima kasih lebih besar di mana tidak berhasil.

Saya menggunakan generator ini selama mencoba: https://www.loremipsum.de/index_e.html

MrGeneration picture MrGeneration pada 3 Jul 2019

Hai,
saya menyalahkan batas dan masalah ini adalah alasan untuk batas tersebut:
https://github.com/zammad/zammad/issues/1390

Dari sisi saya 1,5 MB tidak apa-apa ... Biasanya konten yang lebih besar ditangani dalam lampiran di mana mereka tidak masalah.

Saya mengalami masalah yang sama. Seorang pelanggan mengalami kegagalan impor artikel lain dengan dokumen tertanam yang merusak artikel (1 dari 35k tiket). Tidak ada konten yang dapat digunakan, sepertinya server / klien merusaknya saat mengirim.

Saya akan mengubah pengecualian agar tidak terjadi pada impor, jadi artikel akan diimpor dengan memotongnya menjadi 1,5 MB seperti dalam kasus lain. Kami juga harus memperbaiki batas dari 1,14MB -> 1,5MB. Impor harus berjalan lancar dan mencegah timbulnya pengecualian.

rolfschmidt picture rolfschmidt pada 2 Jun 2020
👍1

Kami memutuskan untuk membatasi 1,5 juta karakter per artikel. Seperti yang ditunjukkan oleh @MrGeneration , UI tidak dapat digunakan setelah jumlah karakter (yang jauh lebih rendah). Itu tidak pernah tentang ukuran sebenarnya dalam hal MB / KB tetapi jumlah karakter. Namun, masalah aslinya sekarang telah diperbaiki dan akan siap digunakan dengan Zammad 3.5 🚀

thorsteneckel picture thorsteneckel pada 10 Jul 2020
👍1
Apakah halaman ini membantu?
0 / 5 - 0 peringkat

Masalah terkait

martinseener picture martinseener  ·  30Komentar
mweinelt picture mweinelt  ·  23Komentar
monotek picture monotek  ·  27Komentar
TeraNovaLP picture TeraNovaLP  ·  31Komentar
PGP
rugk picture rugk  ·  58Komentar