Description: I have a large txt file with some 455000 lines/paragraphs, each with 7 tabs. I copied and pasted its full content into a fresh/empty ODS file, which gave me a Calc file with columns A to H that I saved (file size: 17.5 MB). Now I select and copy cells A1 to A455000. A few seconds later LO crashes. I've tried selecting and copying these cells with several LO versions (from LO 24.2.2 back to 7.4.7.2). Each time after I selected and copied these cells LO crashed a few seconds later. All cells are formatted as text only (i.e. no further formatting is applied). Steps to Reproduce: 1. Select a large number of cells in one column. 2. Copy the selected cells. Actual Results: LO crashes. Expected Results: LO does not crash. Reproducible: Always User Profile Reset: Yes Additional Info: I got a crash report, which I upload here.
Created attachment 193889 [details] crash report
Please upgrade to a more recent LO version (like 7.6.6 or brand new 24.2.2) If you still reproduce this, please apply https://wiki.documentfoundation.org/QA/FirstSteps and above all this part: https://wiki.documentfoundation.org/QA/FirstSteps#Graphics-related_issues_(_Skia_)
(In reply to Julien Nabet from comment #2) > Please upgrade to a more recent LO version (like 7.6.6 or brand new 24.2.2) > > If you still reproduce this, please apply > https://wiki.documentfoundation.org/QA/FirstSteps and above all this part: > https://wiki.documentfoundation.org/QA/FirstSteps#Graphics- > related_issues_(_Skia_) I'm already running LO 24.2.2; I had checked whether older versions (down to 7.4.7.2) also have this issue, which is the case. In About LibreOffice, there is the string "UI render: Skia/Raster; VCL: osx"
I've created and attached a dummy Calc file to illustrate the point. So, using LO 24.2.3.2: 1. Open the file "dummytext.ods" 2. Select the first 10,000 rows and 8 columns by entering A1:H10000 in the Name Box. 3. Copy the selected cells (Cmd+C) into the clipboard. If you repeat these steps but enter A1:H18000 (or any higher number of cells) in step 2, LO will crash. The same happens if you select A1:A150000 (that is, one instead of eight columns). So it seems that the limit of the number of cells that can be selected is somewhere around 144,000 and 150,000 cells only, which is far less than the number of cells I need to select in my current project (455,000 cells).
Created attachment 193942 [details] Calc file with dummy text filling 8x1048576 cells
Version: 24.2.3.2 (X86_64) / LibreOffice Community Build ID: 433d9c2ded56988e8a90e6b2e771ee4e6a5ab2ba CPU threads: 4; OS: macOS 12.7.4; UI render: Skia/Raster; VCL: osx Locale: en-GB (en_GB.UTF-8); UI: en-US Calc: threaded
I can reproduce this on my machine, but only when I quit LibreOffice immediately after copying the cells. Here is what I see in your attached crash log: you are running out of memory when macOS asks LibreOffice for the copied data. In your case, the crash log's memory usage section shows a total of 31.6 GB of memory being used by LibreOffice. Are you running clipboard management software? If yes, that might explain why you see the crash immediately and I don't see it until quitting LibreOffice (clipboard management software usually immediately "pastes" all types of copied data formats immediately after you copy something). OK. Given the above, I think the real bug is that Calc can provide copied cells as a BMP (Windows) image and when quitting or using clipboard management software, every single format that Calc provides is created. My theory is based on the fact that if I copy A1:H10000 in Calc, open a new Writer document, and select HTML format in Writer's Paste Special dialog, LibreOffice does not crash and the data is pasted in the Writer document. But if I paste BMP format in Writer's Paste Special dialog, LibreOffice crashes in the same place as your crash log. So clearly a copy of 80000 cells results in a massive image. Not sure how this can fixed other than removing the BMP format from the formats that copying in Calc provides.
(In reply to Patrick Luby (volunteer) from comment #7) > But if I paste BMP format in Writer's Paste Special dialog, LibreOffice > crashes in the same place as your crash log. So clearly a copy of 80000 > cells results in a massive image. On my machine, copying A1:H10000 will cause LibreOffice to try and create a bitmap that is 1806 x 516820 pixels. That is nearly 1 GB of pixels! Maybe a possible solution is for Calc to not add the BMP format if the number of pixels is more than 1 MB of pixels or something? Not sure how to do that yet.
Just a (semi-random) thought (as a simple user)... What if LO asks the user to release / clean the _relevant_ clipboard space (without touching other clipboard areas that might be occupied by unrelated items, perhaps originated by other programs) when the user attempts to close the program (or maybe, the spreadsheet file)? IIRC, there are some similar comments in some other reports. In that way, only when a "relatively big" amount of the clipboard/RAM was occupied by LO and LO is about to be closed, the user could decide whether he still wants that clipboard area to be available for further actions (i.e. to paste elsewhere), or it should rather be released, for other uses.
(In reply to Patrick Luby (volunteer) from comment #7) > Are you running clipboard management software? If yes, that might explain > why you see the crash immediately and I don't see it until quitting > LibreOffice (clipboard management software usually immediately "pastes" all > types of copied data formats immediately after you copy something). No clipboard manager in use here but several apps that have the ability to store clipboard history. I should have mentioned that after I do Cmd+C on a large number of cells, often the spinning wheel occurs and LO is "not responding" (but not crashing). After a few minutes the spinning wheel disappears and everything is working fine again.
(In reply to Sam from comment #10) > No clipboard manager in use here but several apps that have the ability to > store clipboard history. > > I should have mentioned that after I do Cmd+C on a large number of cells, > often the spinning wheel occurs and LO is "not responding" (but not > crashing). After a few minutes the spinning wheel disappears and everything > is working fine again. Here is what I can do: I can stop the crashing when another application (whether it maintains clipboard history or not) pastes the BMP format and LibreOffice can't allocate enough memory for the image. However, I cannot solve the temporary hanging when another application pastes every single format. That is one big downside of clipboard history applications: they are very convenient but if you copy a large amount of data (and 80000 cells is a lot of data), most clipboard history applications use up a lot of memory very quickly. After all, Calc advertises that the copied cell data can be pasted in 7 different formats and some of those are image formats. I am guessing that this high memory usage is a big contributing factor in the system crashes you have seen in tdf#160644. Don't know how configurable your clipboard history applications are, but hopefully they have some options for pruning their clipboard history after memory usage for the history exceeds some amount. Even better is if your clipboard history applications can be configured to ignore formats that you rarely use. From what I've seen, by default clipboard history applications paste every format available.
I have found a way to stop the crashing when selecting a large number of cells in Calc and I have submitted the following patch: https://gerrit.libreoffice.org/c/core/+/167145 I'll post again when my fix is in a nightly build that you can test.
(In reply to Patrick Luby (volunteer) from comment #11) > However, I cannot solve the temporary hanging when another application > pastes every single format. That is one big downside of clipboard history > applications: they are very convenient but if you copy a large amount of > data (and 80000 cells is a lot of data), most clipboard history applications > use up a lot of memory very quickly. After all, Calc advertises that the > copied cell data can be pasted in 7 different formats and some of those are > image formats. I forgot to mention that I did some testing with my fix for the crashing bug to see what the LibreOffice code is doing during these temporary hangs. Turns out that Calc has 17, not 7, different formats that get copied to the system clipboard. To get an idea of which formats are taking the most time, I opened the attached Calc document, selected all of the cells in the entire sheet, and quit LibreOffice (quitting triggers paste of all formats like most clipboard history applications do by default). It took nearly 10 minutes to copy all 17 formats to the clipboard, but the image formats are no longer part of the problem. The image formats needed less than a second in total. Instead, 85% of the pasting time was spent converting the selected cells to the following 4 formats (sorted from slowest to fastest): public.html application/x-openoffice-embed-source-xml;windows_formatname="Star Embed Source (XML)" public.rtf text/richtext Don't know if its possible to speed those up (I am not familiar with document conversion), but maybe the above list might be useful if any of your clipboard history applications allow certain formats to be ignored when they add to their clipboard history.
Patrick Luby committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/8d9f54165d28d83092667b7bfcd0ee48ade54c87 tdf#160855 fix crash due to Skia's internal maximum pixel limit It will be available in 24.8.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
I have committed the fix for the crash. The fix should be in tomorrow's (07 May 2024) nightly master builds: https://dev-builds.libreoffice.org/daily/master/current.html Note for macOS testers: the nightly master builds install in /Applications/LibreOfficeDev.app. These builds are not codesigned like regular LibreOffice releases so you will need to execute the following Terminal command after installation but before you launch /Applications/LibreOfficeDev: xattr -d com.apple.quarantine /Applications/LibreOfficeDev.app
Patrick Luby committed a patch related to this issue. It has been pushed to "libreoffice-7-6": https://git.libreoffice.org/core/commit/6011796f8d4a1ec81912713f21ec8df5d03769e9 tdf#160855 fix crash due to Skia's internal maximum pixel limit It will be available in 7.6.8. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Patrick Luby committed a patch related to this issue. It has been pushed to "libreoffice-24-2": https://git.libreoffice.org/core/commit/b11c9b5f205907390ef210ae3c1f9f54fee84820 tdf#160855 fix crash due to Skia's internal maximum pixel limit It will be available in 24.2.4. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
(In reply to Patrick Luby (volunteer) from comment #15) > I have committed the fix for the crash. The fix should be in tomorrow's (07 > May 2024) nightly master builds: > > https://dev-builds.libreoffice.org/daily/master/current.html > > Note for macOS testers: the nightly master builds install in > /Applications/LibreOfficeDev.app. These builds are not codesigned like > regular LibreOffice releases so you will need to execute the following > Terminal command after installation but before you launch > /Applications/LibreOfficeDev: > > xattr -d com.apple.quarantine /Applications/LibreOfficeDev.app Thanks! – I downloaded and installed the nightly master build without any problems and run the Terminal command. I started LibreOfficeDev and opened an empty document, which worked. However, the moment I clicked on a file thumbnail in the Recent Documents screen, there was a system crash (blank screen, restart etc.). Attached is the error report that I got after restarting.
Created attachment 194010 [details] error report after system crash see Comment #18
(In reply to Sam from comment #18) > However, the moment I clicked on a file thumbnail in the Recent Documents > screen, there was a system crash (blank screen, restart etc.). Attached is > the error report that I got after restarting. Sorry I now realise the system crash after clicking on a Recent Documents thumbnail also occurs with LO 24.2.3.2. (See attached file "report_2024-05-07b.txt".) So this would be a separate issue. No problem when opening Recent Documents from the File, menu, though, neither in LO 24.2.3.2. nor in 24.8.0.0.alpha0+ (X86_64).
Created attachment 194011 [details] error report after system crash II see Comment #20
With 24.8.0.0.alpha0+ (X86_64), selecting and copying cells A1:H1048576 no longer causes LO to crash. However, LO is not responding for a while, and the Activity Monitor shows LO memory usage of about 2 GB, but after a couple of minutes everything works fine again.
(In reply to Sam from comment #22) > However, LO is not responding for a while, and the Activity Monitor shows LO > memory usage of about 2 GB, but after a couple of minutes everything works > fine again. Sounds like your clipboard history applications are still requesting a copy of every available format (all 17 of them for Calc cells) every time you copy something. Have you tried configuring your clipboard history applications to ignore any formats that you don't need? As I mentioned in comment #13, some of the formats that Calc makes available are very slow to render (e.g. converting the spreadsheet data to HTML format) so if you are able to configure your clipboard history applications to ignore those formats, you can avoid the CPU and memory load of saving formats to history that you don't use. Clipboard history applications are powerful tools, but they do their work by subverting how macOS copy/paste works. Making a copy of everything that any app copies works reasonably well when copying small amounts of data, but it should be no surprise that an application can get overwhelmed when a clipboard history application uses this approach for large amounts of data. BTW, what clipboard history applications are you running on your machine?
Patrick Luby committed a patch related to this issue. It has been pushed to "libreoffice-7-6-7": https://git.libreoffice.org/core/commit/bae8980495d54d235b71938bc313be22d7455585 tdf#160855 fix crash due to Skia's internal maximum pixel limit It will be available in 7.6.7. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
(In reply to Patrick Luby (volunteer) from comment #23) > Sounds like your clipboard history applications are still requesting a copy > of every available format (all 17 of them for Calc cells) every time you > copy something. > > Have you tried configuring your clipboard history applications to ignore any > formats that you don't need? As I mentioned in comment #13, some of the > formats that Calc makes available are very slow to render (e.g. converting > the spreadsheet data to HTML format) so if you are able to configure your > clipboard history applications to ignore those formats, you can avoid the > CPU and memory load of saving formats to history that you don't use. > > BTW, what clipboard history applications are you running on your machine? If I am correct these three apps on my machine fall into the category of (or come close to) clipboard history applications: Keyboard Maestro LaunchBar Typinator I had assumed that these had no effect when copying data into the clipboard (Cmd+C) within LO itself, that is, without me using a shortcut that evokes any of these apps. Is this not the case, then? Since I lack the understanding of how these apps work under the hood I have not yet changed their configuration.
(In reply to Sam from comment #25) > Keyboard Maestro > LaunchBar > Typinator > > I had assumed that these had no effect when copying data into the clipboard > (Cmd+C) within LO itself, that is, without me using a shortcut that evokes > any of these apps. Is this not the case, then? Since I lack the > understanding of how these apps work under the hood I have not yet changed > their configuration. I have not used the above clipboard history applications, but the ones I have seen over the years generally do the following: 1. They monitor for any changes to the system clipboard. 2. When any application, including LibreOffice does copies something, the clipboard history application notices that the system clipboard has been changed and immediately query the system clipboard for what data formats are available 3. The clipboard history application then queries the system clipboard for a copy (i.e. memory) of each format. In contrast, macOS copy and paste is designed around the following process: 1. When any application, including LibreOffice does copies something, it tells the system clipboard that it has data available in a list of formats. But the key here is that no actual data is copied to the system clipboard at this point. 2. Later or never, you paste in the same or a different application. At this point, most applications query the system clipboard for the list of data formats and chooses only either automatically or by the user selecting one from a "paste special" or similar dialog. 3. The pasting application then queries the system clipboard for a copy (i.e. memory) if that one data format. Essentially, clipboard history applications automatically paste every data format of every copy in every application. Whereas, the macOS copy and process only forces the copying application to create the copied data when you actually paste something. In other words, macOS copy and process delays creating a copy until an application wants to paste something. But clipboard history applications force one paste per data format every time something is copied. So there are two things going on in your case. The first is tdf#160644. That bug appears to be your machine is running out of memory. I've got some ideas for using the Activity Monitor application that I will post in tdf#160644 later today to see if one or more clipboard history applications are slowly using up most of your machine's memory. Maybe one of those applications is not trimming their clipboard history when the memory needed for the history gets too large. The second thing going on is that temporary hang that you see. This is happening in step of the above clipboard history applications' process. In the case of Calc, LibreOffice tells the system clipboard that there 17 data formats available every time you copy some cells. Since most applications cannot make sense of either the Excel or Calc spreadsheet format, most of the 17 data formats are more generic data formats like HTML, RTF text, plain text, or images. With the macOS copy and paste process, the idea is that when pasting, the pasting application will select only one of the data formats that it understands and only querey of copy of that format's data. So, Calc only has to convert the selected cells into a different format once. You can see this if you copy in Calc and then paste special in Writer: in Writer's Paste Special dialog you will see a subset of the 17 available Calc data formats that Writer can handle and you can only select one data format. It is true that when you copy cells in Calc and paste within the same spreadsheet, Calc is smart enough to shortcircuit the whole paste process, but the problem is that every time you copy, the clipboard manager applications are immediately pasting every data format into their history. AFAICT, most clipboard history applications don't know which data formats you will want to eventually use so they request all 17 available data formats which causes Calc temporarily hangs while it converts the selected cells to each of the 17 available data formats. The more cells selected, the more conversion work that Calc must do. So what can you do to shorten the temporary hang when copying a large number of cells? With some clipboard history applications, I think you can configure the application to ignore certain data formats. If that is possible, excluding the 4 data formats listed at the end of comment #13 will reduce the conversion work Calc must do by more than 75%. Not sure if that is possible with your clipboard history applications though. Hope that gives a better understanding of what the underlying issues are.