[Unicode]   Common Locale Data Repository : Bug Tracking Home | Site Map | Search
 
Modify

CLDR Ticket #11290(accepted)

Opened 4 months ago

Last modified 3 weeks ago

Survey tool log

Reported by: kristi Owned by: tbishop
Component: survey-backend Data Locale:
Phase: rc Review:
Weeks: Data Xpath:
Xref:

Description

Survey tool log backup keeps bringing down the tool

Attachments

Change History

comment:1 Changed 4 months ago by kristi

  • Owner changed from anybody to tbishop
  • Status changed from new to accepted

comment:2 Changed 4 months ago by emmons

~/tomcat/logs is the location of the logs

comment:3 Changed 4 months ago by tbishop

See also 10295

comment:4 Changed 4 months ago by tbishop

Another ginormous log file: /home/surveytool/tomcat/cldr/exception.log -- 14,253,911,269 bytes

surveytool@CLDRUBS14-PROD2:~/tomcat$ head /home/surveytool/tomcat/cldr/exception.log
!!!***!!! SURVEY_EXCEPTION Including template abbreviated_days.jsp
*** DATE 1526579804978 Thu May 17 17:56:44 UTC 2018
*** REVISION 14017 34 SUBMISSION PRODUCTION
*** UPTIME uptime: 36 sec.
*** LOGSITE org.unicode.cldr.web.SurveyLog.logException(SurveyLog.java:96)
org.unicode.cldr.web.SurveyLog.logException(SurveyLog.java:57)
org.unicode.cldr.web.WebContext.includeFragment(WebContext.java:1813)
org.apache.jsp.WEB_002dINF.tmpl.r_005fsteps_jsp._jspService(r_005fsteps_jsp.java:432)
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
javax.servlet.http.HttpServlet.service(HttpServlet.java:729)

surveytool@CLDRUBS14-PROD2:~/tomcat$ tail /home/surveytool/tomcat/cldr/exception.log
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521)
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096)
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Thread.java:748)
Last edited 4 months ago by tbishop (previous) (diff)

comment:5 Changed 4 months ago by tbishop

Another huge file is /home/surveytool/tomcat/logs/catalina.out which has grown to more than 1GB in less than a month.

cldradmin@st:~$ df -hT /home

Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/sda1      ext4   29G   13G   16G  44% /

cldradmin@st:~$ ls -l /home/surveytool/tomcat/logs
total 1980700
-rw-r--r-- 1 surveytool cldr 166428759 Aug 4 23:59 catalina.2018-08-04.log
-rw-r--r-- 1 surveytool cldr 10357373 Aug 5 02:20 catalina.2018-08-05.log
-rw-r--r-- 1 surveytool cldr 1186514823 Aug 5 04:40 catalina.out
-rw-r--r-- 1 surveytool cldr 47248832 Aug 4 23:59 localhost.2018-08-04.log
-rw-r--r-- 1 surveytool cldr 2940552 Aug 5 01:17 localhost.2018-08-05.log
-rw-r--r-- 1 surveytool cldr 23369809 Jul 27 00:00 localhost_access_log.2018-07-26.txt
-rw-r--r-- 1 surveytool cldr 14907224 Jul 27 23:59 localhost_access_log.2018-07-27.txt
-rw-r--r-- 1 surveytool cldr 5307756 Jul 29 00:00 localhost_access_log.2018-07-28.txt
-rw-r--r-- 1 surveytool cldr 5753515 Jul 30 00:00 localhost_access_log.2018-07-29.txt
-rw-r--r-- 1 surveytool cldr 14208799 Jul 30 23:59 localhost_access_log.2018-07-30.txt
-rw-r--r-- 1 surveytool cldr 4758439 Jul 31 23:59 localhost_access_log.2018-07-31.txt
-rw-r--r-- 1 surveytool cldr 6620413 Aug 2 00:00 localhost_access_log.2018-08-01.txt
-rw-r--r-- 1 surveytool cldr 5123281 Aug 3 00:00 localhost_access_log.2018-08-02.txt
-rw-r--r-- 1 surveytool cldr 3979203 Aug 3 23:59 localhost_access_log.2018-08-03.txt
-rw-r--r-- 1 surveytool cldr 5432247 Aug 5 00:00 localhost_access_log.2018-08-04.txt
-rw-r--r-- 1 surveytool cldr 788689 Aug 5 04:41 localhost_access_log.2018-08-05.txt
-rw-r--r-- 1 root root 52437838 Jul 30 11:26 tomcat.log.1
-rw-r--r-- 1 root root 52429677 Jul 28 12:41 tomcat.log.10
-rw-r--r-- 1 root root 52430148 Jul 30 06:26 tomcat.log.2
-rw-r--r-- 1 root root 52428923 Jul 30 01:54 tomcat.log.3
-rw-r--r-- 1 root root 52435900 Jul 29 20:53 tomcat.log.4
-rw-r--r-- 1 root root 52437562 Jul 29 15:10 tomcat.log.5
-rw-r--r-- 1 root root 52438700 Jul 29 09:46 tomcat.log.6
-rw-r--r-- 1 root root 52429821 Jul 29 04:56 tomcat.log.7
-rw-r--r-- 1 root root 52436076 Jul 28 23:27 tomcat.log.8
-rw-r--r-- 1 root root 52431124 Jul 28 18:02 tomcat.log.9
cldradmin@st:~$ head /home/surveytool/tomcat/logs/catalina.out
10-Jul-2018 13:53:30.909 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version: Apache Tomcat/8.0.30
10-Jul-2018 13:53:30.913 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built: Dec 1 2015 22:30:46 UTC
10-Jul-2018 13:53:30.913 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server number: 8.0.30.0
10-Jul-2018 13:53:30.914 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name: Linux
10-Jul-2018 13:53:30.914 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 4.4.0-71-generic
10-Jul-2018 13:53:30.914 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture: amd64
10-Jul-2018 13:53:30.914 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: /usr/lib/jvm/java-8-oracle/jre
10-Jul-2018 13:53:30.914 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 1.8.0_161-b12
10-Jul-2018 13:53:30.914 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: Oracle Corporation
10-Jul-2018 13:53:30.914 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE: /home/surveytool/tomcat
cldradmin@st:~$ tail /home/surveytool/tomcat/logs/catalina.out
MailSender: processing mail queue
MailSender: processing mail queue
MailSender: processing mail queue
MailSender: processing mail queue
MailSender: processing mail queue
MailSender: processing mail queue
MailSender: processing mail queue
MailSender: processing mail queue
MailSender: processing mail queue
MailSender: processing mail queue

cldradmin@st:~$ sudo rm /home/surveytool/tomcat/logs/catalina.out
cldradmin@st:~$ ls -l /home/surveytool/tomcat/cldr/exception.log
-rw-r--r-- 1 surveytool cldr 234185468 Aug 5 01:17 /home/surveytool/tomcat/cldr/exception.log
cldradmin@st:~$ head /home/surveytool/tomcat/cldr/exception.log
!!***!!! SURVEY_EXCEPTION Including template eras.jsp
...
cldradmin@st:~$ sudo rm /home/surveytool/tomcat/cldr/exception.log

comment:6 Changed 4 months ago by tbishop

Plan something like this. Create file named: /etc/logrotate.d/surveytool-logs

Contents of that file:

/home/surveytool/tomcat/cldr/exception.log {
    daily
    missingok
    rotate 14
    compress
    notifempty
    dateext
    create 0640 surveytool cldr
}

Plus similar entries, maybe in the same file, for /home/surveytool/tomcat/logs/catalina.out and maybe everything in the logs folder.

comment:7 Changed 4 months ago by tbishop

I created a file /etc/logrotate.d/surveytool-logs with these contents:

/home/surveytool/tomcat/cldr/exception.log /home/surveytool/tomcat/logs/catalina.out {
    daily
    rotate 7
    compress
    missingok
    notifempty
    dateext
    create 0640 surveytool cldr
}

I did:

sudo chown root:root /etc/logrotate.d/surveytool-logs
sudo chmod 644 /etc/logrotate.d/surveytool-logs

Still to do:

  • monitor operation for the next few days
  • rotate other files in /home/surveytool/tomcat/logs
  • rsync to a different location

comment:8 Changed 3 months ago by tbishop

Log rotation appears to be working now for /home/surveytool/tomcat/cldr/exception.log; we have newly-created exception.log-20180807.gz.

However, there is no /home/surveytool/tomcat/logs/catalina.out yet, nor any rotated version of it. Maybe it's necessary to restart tomcat, possibly with prerotate and postrotate sections? For example:

prerotate

/home/surveytool/tomcat/bin/shutdown.sh

endscript
postrotate

/home/surveytool/tomcat/bin/startup.sh

endscript

Alternatively it might suffice to add "copytruncate". "...It can be used when some program cannot be told to close its logfile and thus might continue writing (appending) to the previous log file forever. Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost...."

Last edited 3 months ago by tbishop (previous) (diff)

comment:9 Changed 3 months ago by tbishop

I added copytruncate, and ran shutdown.sh and startup.sh. A new catalina.out appeared. I'll keep an eye on it.

comment:10 Changed 3 months ago by tbishop

Update:

cldradmin@st:~$ ls -l /home/surveytool/tomcat/logs
...
-rw-r--r-- 1 surveytool cldr 126586766 Aug 9 14:01 catalina.out
-rw-r--r-- 1 surveytool cldr 2048692 Aug 9 06:24 catalina.out-20180809.gz
...

cldradmin@st:~$ ls -l /home/surveytool/tomcat/cldr
...
-rw-r----- 1 surveytool cldr 94294574 Aug 9 14:02 exception.log
-rw-r--r-- 1 surveytool cldr 3012039 Aug 7 06:24 exception.log-20180807.gz
-rw-r----- 1 surveytool cldr 3103955 Aug 8 06:24 exception.log-20180808.gz
-rw-r----- 1 surveytool cldr 3661160 Aug 9 06:24 exception.log-20180809.gz
...

comment:11 Changed 3 months ago by tbishop

Usage of /: 50.6% of 28.80GB
cldradmin@st:~$ ls -l /home/surveytool/tomcat/logs
...
-rw-r--r-- 1 surveytool cldr 75866227 Aug 14 12:44 catalina.out
-rw-r--r-- 1 surveytool cldr 2048692 Aug 9 06:24 catalina.out-20180809.gz
-rw-r--r-- 1 surveytool cldr 5406026 Aug 10 06:24 catalina.out-20180810.gz
-rw-r--r-- 1 surveytool cldr 4621011 Aug 11 06:24 catalina.out-20180811.gz
-rw-r--r-- 1 surveytool cldr 3815213 Aug 12 06:24 catalina.out-20180812.gz
-rw-r--r-- 1 surveytool cldr 4077717 Aug 13 06:24 catalina.out-20180813.gz
-rw-r--r-- 1 surveytool cldr 3459016 Aug 14 06:24 catalina.out-20180814.gz
...
cldradmin@st:~$ ls -l /home/surveytool/tomcat/cldr
...
-rw-r----- 1 surveytool cldr 56673641 Aug 14 12:45 exception.log
-rw-r----- 1 surveytool cldr 3103955 Aug 8 06:24 exception.log-20180808.gz
-rw-r----- 1 surveytool cldr 3661160 Aug 9 06:24 exception.log-20180809.gz
-rw-r----- 1 surveytool cldr 3371090 Aug 10 06:24 exception.log-20180810.gz
-rw-r----- 1 surveytool cldr 2899152 Aug 11 06:24 exception.log-20180811.gz
-rw-r----- 1 surveytool cldr 2469279 Aug 12 06:24 exception.log-20180812.gz
-rw-r----- 1 surveytool cldr 2530489 Aug 13 06:24 exception.log-20180813.gz
-rw-r----- 1 surveytool cldr 2198054 Aug 14 06:24 exception.log-20180814.gz
...

comment:12 Changed 3 months ago by tbishop

Disk is now 58.6% full.

The rotations I've set up produce compressed files with .gz extension, like catalina.out-20180822.gz and exception.log-20180822.gz. There are also other files like catalina.2018-08-22.log and I'm not sure how they're being produced. I'll look into that...

cldradmin@st:~$ ls -l /home/surveytool/tomcat/logs
total 5917140
-rw-r--r-- 1 surveytool cldr 166428759 Aug  4 23:59 catalina.2018-08-04.log
-rw-r--r-- 1 surveytool cldr  96117800 Aug  5 23:59 catalina.2018-08-05.log
-rw-r--r-- 1 surveytool cldr 131805234 Aug  6 23:59 catalina.2018-08-06.log
-rw-r--r-- 1 surveytool cldr 181957255 Aug  7 23:59 catalina.2018-08-07.log
-rw-r--r-- 1 surveytool cldr 290848340 Aug  8 23:59 catalina.2018-08-08.log
-rw-r--r-- 1 surveytool cldr 280710016 Aug  9 23:59 catalina.2018-08-09.log
-rw-r--r-- 1 surveytool cldr 247057885 Aug 10 23:59 catalina.2018-08-10.log
-rw-r--r-- 1 surveytool cldr 235827860 Aug 11 23:59 catalina.2018-08-11.log
-rw-r--r-- 1 surveytool cldr 245835524 Aug 12 23:59 catalina.2018-08-12.log
-rw-r--r-- 1 surveytool cldr 223579646 Aug 13 23:59 catalina.2018-08-13.log
-rw-r--r-- 1 surveytool cldr 222871850 Aug 14 23:59 catalina.2018-08-14.log
-rw-r--r-- 1 surveytool cldr 165621830 Aug 15 23:59 catalina.2018-08-15.log
-rw-r--r-- 1 surveytool cldr 238482653 Aug 16 23:59 catalina.2018-08-16.log
-rw-r--r-- 1 surveytool cldr 237827365 Aug 17 23:59 catalina.2018-08-17.log
-rw-r--r-- 1 surveytool cldr 246487105 Aug 18 23:59 catalina.2018-08-18.log
-rw-r--r-- 1 surveytool cldr 253633565 Aug 19 23:59 catalina.2018-08-19.log
-rw-r--r-- 1 surveytool cldr 244603690 Aug 20 23:59 catalina.2018-08-20.log
-rw-r--r-- 1 surveytool cldr 245140939 Aug 21 23:59 catalina.2018-08-21.log
-rw-r--r-- 1 surveytool cldr 119864658 Aug 22 12:03 catalina.2018-08-22.log
-rw-r--r-- 1 surveytool cldr  71767156 Aug 22 12:03 catalina.out
-rw-r--r-- 1 surveytool cldr   3373263 Aug 16 06:25 catalina.out-20180816.gz
-rw-r--r-- 1 surveytool cldr   3901752 Aug 17 06:25 catalina.out-20180817.gz
-rw-r--r-- 1 surveytool cldr   4085568 Aug 18 06:24 catalina.out-20180818.gz
-rw-r--r-- 1 surveytool cldr   4100526 Aug 19 06:24 catalina.out-20180819.gz
-rw-r--r-- 1 surveytool cldr   4141927 Aug 20 06:24 catalina.out-20180820.gz
-rw-r--r-- 1 surveytool cldr   4095461 Aug 21 06:24 catalina.out-20180821.gz
-rw-r--r-- 1 surveytool cldr   5175775 Aug 22 06:24 catalina.out-20180822.gz
-rw-r--r-- 1 surveytool cldr         0 Aug  6 13:31 host-manager.2018-08-06.log
-rw-r--r-- 1 surveytool cldr         0 Aug  8 21:13 host-manager.2018-08-08.log
-rw-r--r-- 1 surveytool cldr         0 Aug  9 01:32 host-manager.2018-08-09.log
-rw-r--r-- 1 surveytool cldr         0 Aug 10 15:24 host-manager.2018-08-10.log
-rw-r--r-- 1 surveytool cldr  47248832 Aug  4 23:59 localhost.2018-08-04.log
-rw-r--r-- 1 surveytool cldr  27257522 Aug  5 23:59 localhost.2018-08-05.log
-rw-r--r-- 1 surveytool cldr  36826006 Aug  6 23:59 localhost.2018-08-06.log
-rw-r--r-- 1 surveytool cldr  49874944 Aug  7 23:59 localhost.2018-08-07.log
-rw-r--r-- 1 surveytool cldr  80008455 Aug  8 23:59 localhost.2018-08-08.log
-rw-r--r-- 1 surveytool cldr  78422142 Aug  9 23:59 localhost.2018-08-09.log
-rw-r--r-- 1 surveytool cldr  68537469 Aug 10 23:59 localhost.2018-08-10.log
-rw-r--r-- 1 surveytool cldr  66953824 Aug 11 23:59 localhost.2018-08-11.log
-rw-r--r-- 1 surveytool cldr  69795612 Aug 12 23:59 localhost.2018-08-12.log
-rw-r--r-- 1 surveytool cldr  63476623 Aug 13 23:59 localhost.2018-08-13.log
-rw-r--r-- 1 surveytool cldr  63275734 Aug 14 23:59 localhost.2018-08-14.log
-rw-r--r-- 1 surveytool cldr  47020506 Aug 15 23:59 localhost.2018-08-15.log
-rw-r--r-- 1 surveytool cldr  67681065 Aug 16 23:59 localhost.2018-08-16.log
-rw-r--r-- 1 surveytool cldr  67492029 Aug 17 23:59 localhost.2018-08-17.log
-rw-r--r-- 1 surveytool cldr  69990742 Aug 18 23:59 localhost.2018-08-18.log
-rw-r--r-- 1 surveytool cldr  72012358 Aug 19 23:59 localhost.2018-08-19.log
-rw-r--r-- 1 surveytool cldr  69432902 Aug 20 23:59 localhost.2018-08-20.log
-rw-r--r-- 1 surveytool cldr  69595470 Aug 21 23:59 localhost.2018-08-21.log
-rw-r--r-- 1 surveytool cldr  34028571 Aug 22 12:03 localhost.2018-08-22.log
-rw-r--r-- 1 surveytool cldr  23369809 Jul 27 00:00 localhost_access_log.2018-07-26.txt
-rw-r--r-- 1 surveytool cldr  14907224 Jul 27 23:59 localhost_access_log.2018-07-27.txt
-rw-r--r-- 1 surveytool cldr   5307756 Jul 29 00:00 localhost_access_log.2018-07-28.txt
-rw-r--r-- 1 surveytool cldr   5753515 Jul 30 00:00 localhost_access_log.2018-07-29.txt
-rw-r--r-- 1 surveytool cldr  14208799 Jul 30 23:59 localhost_access_log.2018-07-30.txt
-rw-r--r-- 1 surveytool cldr   4758439 Jul 31 23:59 localhost_access_log.2018-07-31.txt
-rw-r--r-- 1 surveytool cldr   6620413 Aug  2 00:00 localhost_access_log.2018-08-01.txt
-rw-r--r-- 1 surveytool cldr   5123281 Aug  3 00:00 localhost_access_log.2018-08-02.txt
-rw-r--r-- 1 surveytool cldr   3979203 Aug  3 23:59 localhost_access_log.2018-08-03.txt
-rw-r--r-- 1 surveytool cldr   5432247 Aug  5 00:00 localhost_access_log.2018-08-04.txt
-rw-r--r-- 1 surveytool cldr   4624260 Aug  5 23:59 localhost_access_log.2018-08-05.txt
-rw-r--r-- 1 surveytool cldr   4885326 Aug  7 00:00 localhost_access_log.2018-08-06.txt
-rw-r--r-- 1 surveytool cldr   5573419 Aug  8 00:00 localhost_access_log.2018-08-07.txt
-rw-r--r-- 1 surveytool cldr   9655062 Aug  8 23:59 localhost_access_log.2018-08-08.txt
-rw-r--r-- 1 surveytool cldr   8612669 Aug  9 23:59 localhost_access_log.2018-08-09.txt
-rw-r--r-- 1 surveytool cldr   4951239 Aug 11 00:00 localhost_access_log.2018-08-10.txt
-rw-r--r-- 1 surveytool cldr   3685333 Aug 11 23:59 localhost_access_log.2018-08-11.txt
-rw-r--r-- 1 surveytool cldr   3611199 Aug 12 23:59 localhost_access_log.2018-08-12.txt
-rw-r--r-- 1 surveytool cldr   5979741 Aug 13 23:59 localhost_access_log.2018-08-13.txt
-rw-r--r-- 1 surveytool cldr   7939092 Aug 14 23:59 localhost_access_log.2018-08-14.txt
-rw-r--r-- 1 surveytool cldr   8136335 Aug 16 00:00 localhost_access_log.2018-08-15.txt
-rw-r--r-- 1 surveytool cldr   8230927 Aug 17 00:00 localhost_access_log.2018-08-16.txt
-rw-r--r-- 1 surveytool cldr   8080395 Aug 18 00:00 localhost_access_log.2018-08-17.txt
-rw-r--r-- 1 surveytool cldr   4937653 Aug 19 00:00 localhost_access_log.2018-08-18.txt
-rw-r--r-- 1 surveytool cldr   5077140 Aug 20 00:00 localhost_access_log.2018-08-19.txt
-rw-r--r-- 1 surveytool cldr   6743788 Aug 21 00:00 localhost_access_log.2018-08-20.txt
-rw-r--r-- 1 surveytool cldr  13581155 Aug 22 00:00 localhost_access_log.2018-08-21.txt
-rw-r--r-- 1 surveytool cldr   6203208 Aug 22 12:03 localhost_access_log.2018-08-22.txt
-rw-r--r-- 1 surveytool cldr         0 Aug  6 13:31 manager.2018-08-06.log
-rw-r--r-- 1 surveytool cldr         0 Aug  8 21:13 manager.2018-08-08.log
-rw-r--r-- 1 surveytool cldr         0 Aug  9 01:32 manager.2018-08-09.log
-rw-r--r-- 1 surveytool cldr         0 Aug 10 15:24 manager.2018-08-10.log
-rw-r--r-- 1 root       root     78105 Aug  9 12:42 tomcat.log
-rw-r--r-- 1 root       root  52437838 Jul 30 11:26 tomcat.log.1
-rw-r--r-- 1 root       root  52429677 Jul 28 12:41 tomcat.log.10
-rw-r--r-- 1 root       root  52430148 Jul 30 06:26 tomcat.log.2
-rw-r--r-- 1 root       root  52428923 Jul 30 01:54 tomcat.log.3
-rw-r--r-- 1 root       root  52435900 Jul 29 20:53 tomcat.log.4
-rw-r--r-- 1 root       root  52437562 Jul 29 15:10 tomcat.log.5
-rw-r--r-- 1 root       root  52438700 Jul 29 09:46 tomcat.log.6
-rw-r--r-- 1 root       root  52429821 Jul 29 04:56 tomcat.log.7
-rw-r--r-- 1 root       root  52436076 Jul 28 23:27 tomcat.log.8
-rw-r--r-- 1 root       root  52431124 Jul 28 18:02 tomcat.log.9

cldradmin@st:~$ ls -l /home/surveytool/tomcat/cldr
total 118444
drwxr-xr-x  4 surveytool cldr     4096 Dec 16  2015 28
drwxr-xr-x  4 surveytool cldr     4096 Mar 30  2016 29
drwxr-xr-x  4 surveytool cldr     4096 Oct 20  2016 30
drwxr-xr-x  4 surveytool cldr     4096 Mar 31  2017 31
drwxr-xr-x  4 surveytool cldr     4096 Nov  9  2017 32
drwxr-xr-x  4 surveytool cldr     4096 Apr 26 19:55 33
-rw-r--r--  1 surveytool cldr      656 Dec 16  2015 admin.html
drwxr-xr-x  2 surveytool cldr     4096 Sep  7  2017 cache
drwxr-xr-x  5 surveytool cldr     4096 Dec 16  2015 cldrdb
-rw-r--r--  1 surveytool cldr 49900742 Aug 13 09:06 cldrmail.log
-rw-r--r--  1 surveytool cldr      703 Aug 10 15:23 cldr.properties
-rw-r--r--  1 surveytool cldr      871 Aug  5  2016 cldr.properties~
-rw-r--r--  1 surveytool cldr     1245 May 17 17:26 cldr.properties___
-rw-r--r--  1 surveytool cldr      726 May 28 16:31 cldr.properties.backup
-rw-r--r--  1 surveytool cldr       18 Aug  9 12:42 currev.properties
-rw-r-----  1 surveytool cldr 54226213 Aug 22 12:09 exception.log
-rw-r-----  1 surveytool cldr  1908104 Aug 16 06:25 exception.log-20180816.gz
-rw-r-----  1 surveytool cldr  2452728 Aug 17 06:25 exception.log-20180817.gz
-rw-r-----  1 surveytool cldr  2504962 Aug 18 06:24 exception.log-20180818.gz
-rw-r-----  1 surveytool cldr  2530740 Aug 19 06:24 exception.log-20180819.gz
-rw-r-----  1 surveytool cldr  2645567 Aug 20 06:24 exception.log-20180820.gz
-rw-r-----  1 surveytool cldr  2482938 Aug 21 06:24 exception.log-20180821.gz
-rw-r-----  1 surveytool cldr  2513445 Aug 22 06:24 exception.log-20180822.gz
drwxr-xr-x 10 surveytool cldr     4096 Jul 13  2016 vetdata

comment:13 Changed 3 months ago by tbishop

server.xml has "localhost_access_log":

        <!-- Access log processes all example.
             Documentation at: /docs/config/valve.html
             Note: The pattern used is equivalent to using pattern="common" -->
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" pattern="%h %l %u %t &quot;%r&quot; %s %b" prefix="localhost_access_log" suffix=".txt"/>

Here's info about tomcat logs:

https://stackoverflow.com/questions/40935536/whats-the-difference-between-localhost-log-catalina-log-manager-log-host-man

https://tomcat.apache.org/tomcat-8.5-doc/logging.html

comment:14 Changed 3 months ago by tbishop

cldradmin@st:~$ df -hT /home
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/sda1      ext4   29G   20G  8.2G  71% /

The disk is getting full again. I'm deleting a bunch of the uncompressed logs, bringing used down to 46 percent.

Much of what is filling the logs is due to this function in JspWebContext.java:

   /**
     * Close the main <form> iff it is already open. Throws an error if it was
     * not open.
     */
    public void closeMainForm() {
        if (mainFormState == MainFormState.OPEN) {
            String nextStep = (String) this.get("nextStep");
            if (nextStep != null) {
                this.println("Not implemented:  nextStep");
                STFactory.unimp();

That STFactory.unimp() throws an exception, stack trace goes into logs... I've made a new ticket 11396 for the closeMainForm bug.

Last edited 3 months ago by tbishop (previous) (diff)

comment:15 Changed 2 months ago by tbishop

Further tomcat documentation:

https://tomcat.apache.org/tomcat-8.0-doc/config/valve.html

fileDateFormat
Allows a customized timestamp in the access log file name. The file is rotated whenever the formatted timestamp changes. The default value is yyyy-MM-dd. If you wish to rotate every hour, then set this value to yyyy-MM-dd.HH. The date format will always be localized using the locale en_US.

maxDays
The maximum number of days rotated access logs will be retained for before being deleted. If not specified, the default value of -1 will be used which means never delete old files.

prefix
The prefix added to the start of each log file's name. If not specified, the default value is "access_log".

rotatable
Flag to determine if log rotation should occur. If set to false, then this file is never rotated and fileDateFormat is ignored. Default value: true

None of those seem to be in /home/surveytool/tomcat/conf/survey.xml. Do the default values account for names like "localhost_access_log.2018-08-22.txt"?

In server.xml we have prefix="localhost_access_log" suffix=".txt" here:

        <!-- Access log processes all example.
             Documentation at: /docs/config/valve.html
             Note: The pattern used is equivalent to using pattern="common" -->
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log" suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

Should we add a setting for maxDays?

In /home/surveytool/tomcat/conf/logging.properties these are included:

1catalina.org.apache.juli.AsyncFileHandler.level = FINE
1catalina.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs
1catalina.org.apache.juli.AsyncFileHandler.prefix = catalina.

2localhost.org.apache.juli.AsyncFileHandler.level = FINE
2localhost.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs
2localhost.org.apache.juli.AsyncFileHandler.prefix = localhost.

3manager.org.apache.juli.AsyncFileHandler.level = FINE
3manager.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs
3manager.org.apache.juli.AsyncFileHandler.prefix = manager.

4host-manager.org.apache.juli.AsyncFileHandler.level = FINE
4host-manager.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs
4host-manager.org.apache.juli.AsyncFileHandler.prefix = host-manager.

Last edited 2 months ago by tbishop (previous) (diff)

comment:16 Changed 7 weeks ago by tbishop

It would help a lot to compress those logs that aren't already getting compressed. That could be added to the current lograte script.

I'm setting up to rsync old files to corp.unicode.org for longer-term storage.

Welcome to Ubuntu 16.04.4 LTS (GNU/Linux 4.4.0-71-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Fri Sep 28 14:24:45 UTC 2018

  System load:  0.15               Processes:           252
  Usage of /:   58.9% of 28.80GB   Users logged in:     0
  Memory usage: 13%                IP address for eth0: 100.72.138.74
  Swap usage:   0%

 ...

cldradmin@st:~$ df -hT /home
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/sda1      ext4   29G   17G   11G  62% /
cldradmin@st:~$ ls -l /home/surveytool/tomcat/logs
total 5626480
-rw-r--r-- 1 surveytool cldr 160025585 Sep 10 23:59 catalina.2018-09-10.log
-rw-r--r-- 1 surveytool cldr 255328233 Sep 11 23:59 catalina.2018-09-11.log
-rw-r--r-- 1 surveytool cldr 235939801 Sep 12 23:59 catalina.2018-09-12.log
-rw-r--r-- 1 surveytool cldr 245722885 Sep 13 23:59 catalina.2018-09-13.log
-rw-r--r-- 1 surveytool cldr 240467148 Sep 14 23:59 catalina.2018-09-14.log
-rw-r--r-- 1 surveytool cldr 157950113 Sep 15 23:59 catalina.2018-09-15.log
-rw-r--r-- 1 surveytool cldr 217225271 Sep 16 23:59 catalina.2018-09-16.log
-rw-r--r-- 1 surveytool cldr 204271762 Sep 17 23:59 catalina.2018-09-17.log
-rw-r--r-- 1 surveytool cldr 174246018 Sep 18 23:59 catalina.2018-09-18.log
-rw-r--r-- 1 surveytool cldr 167353840 Sep 19 23:59 catalina.2018-09-19.log
-rw-r--r-- 1 surveytool cldr 174607483 Sep 20 23:59 catalina.2018-09-20.log
-rw-r--r-- 1 surveytool cldr 144691558 Sep 21 23:59 catalina.2018-09-21.log
-rw-r--r-- 1 surveytool cldr 117852532 Sep 22 23:59 catalina.2018-09-22.log
-rw-r--r-- 1 surveytool cldr 121056072 Sep 23 23:59 catalina.2018-09-23.log
-rw-r--r-- 1 surveytool cldr 129983594 Sep 24 23:58 catalina.2018-09-24.log
-rw-r--r-- 1 surveytool cldr  93109608 Sep 25 23:57 catalina.2018-09-25.log
-rw-r--r-- 1 surveytool cldr 165671991 Sep 26 23:59 catalina.2018-09-26.log
-rw-r--r-- 1 surveytool cldr 184709846 Sep 27 23:59 catalina.2018-09-27.log
-rw-r--r-- 1 surveytool cldr 117714801 Sep 28 14:27 catalina.2018-09-28.log
-rw-r--r-- 1 surveytool cldr  82677074 Sep 28 14:27 catalina.out
-rw-r--r-- 1 surveytool cldr   2076779 Sep 22 06:24 catalina.out-20180922.gz
-rw-r--r-- 1 surveytool cldr   1948986 Sep 23 06:25 catalina.out-20180923.gz
-rw-r--r-- 1 surveytool cldr   2110365 Sep 24 06:25 catalina.out-20180924.gz
-rw-r--r-- 1 surveytool cldr   2189478 Sep 25 06:24 catalina.out-20180925.gz
-rw-r--r-- 1 surveytool cldr   1717663 Sep 26 06:24 catalina.out-20180926.gz
-rw-r--r-- 1 surveytool cldr   3027424 Sep 27 06:24 catalina.out-20180927.gz
-rw-r--r-- 1 surveytool cldr   3215163 Sep 28 06:24 catalina.out-20180928.gz
-rw-r--r-- 1 surveytool cldr         0 Aug  6 13:31 host-manager.2018-08-06.log
-rw-r--r-- 1 surveytool cldr         0 Aug  8 21:13 host-manager.2018-08-08.log
-rw-r--r-- 1 surveytool cldr         0 Aug  9 01:32 host-manager.2018-08-09.log
-rw-r--r-- 1 surveytool cldr         0 Aug 10 15:24 host-manager.2018-08-10.log
-rw-r--r-- 1 surveytool cldr  75761153 Aug 30 23:59 localhost.2018-08-30.log
-rw-r--r-- 1 surveytool cldr  67640926 Aug 31 23:59 localhost.2018-08-31.log
-rw-r--r-- 1 surveytool cldr  72388416 Sep  1 23:59 localhost.2018-09-01.log
-rw-r--r-- 1 surveytool cldr  65834995 Sep  2 23:59 localhost.2018-09-02.log
-rw-r--r-- 1 surveytool cldr  47227181 Sep  3 23:59 localhost.2018-09-03.log
-rw-r--r-- 1 surveytool cldr  72419638 Sep  4 23:59 localhost.2018-09-04.log
-rw-r--r-- 1 surveytool cldr  71749103 Sep  5 23:59 localhost.2018-09-05.log
-rw-r--r-- 1 surveytool cldr  74147732 Sep  6 23:59 localhost.2018-09-06.log
-rw-r--r-- 1 surveytool cldr  70332350 Sep  7 23:59 localhost.2018-09-07.log
-rw-r--r-- 1 surveytool cldr  69681420 Sep  8 23:59 localhost.2018-09-08.log
-rw-r--r-- 1 surveytool cldr  66702569 Sep  9 23:59 localhost.2018-09-09.log
-rw-r--r-- 1 surveytool cldr  45415861 Sep 10 23:59 localhost.2018-09-10.log
-rw-r--r-- 1 surveytool cldr  72483650 Sep 11 23:59 localhost.2018-09-11.log
-rw-r--r-- 1 surveytool cldr  66982156 Sep 12 23:59 localhost.2018-09-12.log
-rw-r--r-- 1 surveytool cldr  69758690 Sep 13 23:59 localhost.2018-09-13.log
-rw-r--r-- 1 surveytool cldr  68268025 Sep 14 23:59 localhost.2018-09-14.log
-rw-r--r-- 1 surveytool cldr  44840805 Sep 15 23:59 localhost.2018-09-15.log
-rw-r--r-- 1 surveytool cldr  61668686 Sep 16 23:59 localhost.2018-09-16.log
-rw-r--r-- 1 surveytool cldr  57983036 Sep 17 23:59 localhost.2018-09-17.log
-rw-r--r-- 1 surveytool cldr  49466806 Sep 18 23:59 localhost.2018-09-18.log
-rw-r--r-- 1 surveytool cldr  47510830 Sep 19 23:59 localhost.2018-09-19.log
-rw-r--r-- 1 surveytool cldr  49546593 Sep 20 23:59 localhost.2018-09-20.log
-rw-r--r-- 1 surveytool cldr  41074719 Sep 21 23:59 localhost.2018-09-21.log
-rw-r--r-- 1 surveytool cldr  33454394 Sep 22 23:59 localhost.2018-09-22.log
-rw-r--r-- 1 surveytool cldr  34363911 Sep 23 23:59 localhost.2018-09-23.log
-rw-r--r-- 1 surveytool cldr  36887587 Sep 24 23:58 localhost.2018-09-24.log
-rw-r--r-- 1 surveytool cldr  26426960 Sep 25 23:57 localhost.2018-09-25.log
-rw-r--r-- 1 surveytool cldr  47014247 Sep 26 23:59 localhost.2018-09-26.log
-rw-r--r-- 1 surveytool cldr  52430756 Sep 27 23:59 localhost.2018-09-27.log
-rw-r--r-- 1 surveytool cldr  33391677 Sep 28 14:27 localhost.2018-09-28.log
-rw-r--r-- 1 surveytool cldr   3882151 Sep  1 23:59 localhost_access_log.2018-09-01.txt
-rw-r--r-- 1 surveytool cldr   3564490 Sep  2 23:59 localhost_access_log.2018-09-02.txt
-rw-r--r-- 1 surveytool cldr   3710090 Sep  3 23:59 localhost_access_log.2018-09-03.txt
-rw-r--r-- 1 surveytool cldr   4501578 Sep  5 00:00 localhost_access_log.2018-09-04.txt
-rw-r--r-- 1 surveytool cldr   5364117 Sep  5 23:59 localhost_access_log.2018-09-05.txt
-rw-r--r-- 1 surveytool cldr   5768738 Sep  6 23:59 localhost_access_log.2018-09-06.txt
-rw-r--r-- 1 surveytool cldr   5173747 Sep  8 00:00 localhost_access_log.2018-09-07.txt
-rw-r--r-- 1 surveytool cldr   3827063 Sep  9 00:00 localhost_access_log.2018-09-08.txt
-rw-r--r-- 1 surveytool cldr   3767167 Sep  9 23:59 localhost_access_log.2018-09-09.txt
-rw-r--r-- 1 surveytool cldr   4439002 Sep 11 00:00 localhost_access_log.2018-09-10.txt
-rw-r--r-- 1 surveytool cldr   6119746 Sep 11 23:59 localhost_access_log.2018-09-11.txt
-rw-r--r-- 1 surveytool cldr   5494507 Sep 12 23:59 localhost_access_log.2018-09-12.txt
-rw-r--r-- 1 surveytool cldr   6560500 Sep 14 00:00 localhost_access_log.2018-09-13.txt
-rw-r--r-- 1 surveytool cldr   5732625 Sep 15 00:00 localhost_access_log.2018-09-14.txt
-rw-r--r-- 1 surveytool cldr   4496193 Sep 16 00:00 localhost_access_log.2018-09-15.txt
-rw-r--r-- 1 surveytool cldr   5055806 Sep 17 00:00 localhost_access_log.2018-09-16.txt
-rw-r--r-- 1 surveytool cldr   5545152 Sep 18 00:00 localhost_access_log.2018-09-17.txt
-rw-r--r-- 1 surveytool cldr   4705100 Sep 19 00:00 localhost_access_log.2018-09-18.txt
-rw-r--r-- 1 surveytool cldr   6262742 Sep 20 00:00 localhost_access_log.2018-09-19.txt
-rw-r--r-- 1 surveytool cldr   5780158 Sep 21 00:00 localhost_access_log.2018-09-20.txt
-rw-r--r-- 1 surveytool cldr   5476740 Sep 22 00:00 localhost_access_log.2018-09-21.txt
-rw-r--r-- 1 surveytool cldr   4174759 Sep 23 00:00 localhost_access_log.2018-09-22.txt
-rw-r--r-- 1 surveytool cldr   4692402 Sep 23 23:59 localhost_access_log.2018-09-23.txt
-rw-r--r-- 1 surveytool cldr   6629973 Sep 25 00:00 localhost_access_log.2018-09-24.txt
-rw-r--r-- 1 surveytool cldr   4287355 Sep 26 00:00 localhost_access_log.2018-09-25.txt
-rw-r--r-- 1 surveytool cldr   5026854 Sep 27 00:00 localhost_access_log.2018-09-26.txt
-rw-r--r-- 1 surveytool cldr   4361071 Sep 28 00:00 localhost_access_log.2018-09-27.txt
-rw-r--r-- 1 surveytool cldr   2401907 Sep 28 14:27 localhost_access_log.2018-09-28.txt
-rw-r--r-- 1 surveytool cldr         0 Aug  6 13:31 manager.2018-08-06.log
-rw-r--r-- 1 surveytool cldr         0 Aug  8 21:13 manager.2018-08-08.log
-rw-r--r-- 1 surveytool cldr         0 Aug  9 01:32 manager.2018-08-09.log
-rw-r--r-- 1 surveytool cldr         0 Aug 10 15:24 manager.2018-08-10.log
-rw-r--r-- 1 root       root     78105 Aug  9 12:42 tomcat.log
-rw-r--r-- 1 root       root  52437838 Jul 30 11:26 tomcat.log.1
-rw-r--r-- 1 root       root  52429677 Jul 28 12:41 tomcat.log.10
-rw-r--r-- 1 root       root  52430148 Jul 30 06:26 tomcat.log.2
-rw-r--r-- 1 root       root  52428923 Jul 30 01:54 tomcat.log.3
-rw-r--r-- 1 root       root  52435900 Jul 29 20:53 tomcat.log.4
-rw-r--r-- 1 root       root  52437562 Jul 29 15:10 tomcat.log.5
-rw-r--r-- 1 root       root  52438700 Jul 29 09:46 tomcat.log.6
-rw-r--r-- 1 root       root  52429821 Jul 29 04:56 tomcat.log.7
-rw-r--r-- 1 root       root  52436076 Jul 28 23:27 tomcat.log.8
-rw-r--r-- 1 root       root  52431124 Jul 28 18:02 tomcat.log.9
cldradmin@st:~$  ls -l /home/surveytool/tomcat/cldr
total 118084
drwxr-xr-x  4 surveytool cldr     4096 Dec 16  2015 28
drwxr-xr-x  4 surveytool cldr     4096 Mar 30  2016 29
drwxr-xr-x  4 surveytool cldr     4096 Oct 20  2016 30
drwxr-xr-x  4 surveytool cldr     4096 Mar 31  2017 31
drwxr-xr-x  4 surveytool cldr     4096 Nov  9  2017 32
drwxr-xr-x  4 surveytool cldr     4096 Apr 26 19:55 33
-rw-r--r--  1 surveytool cldr      656 Dec 16  2015 admin.html
drwxr-xr-x  2 surveytool cldr     4096 Sep  7  2017 cache
drwxr-xr-x  5 surveytool cldr     4096 Dec 16  2015 cldrdb
-rw-r--r--  1 surveytool cldr 49902602 Sep 17 01:10 cldrmail.log
-rw-r--r--  1 surveytool cldr      703 Aug 10 15:23 cldr.properties
-rw-r--r--  1 surveytool cldr      871 Aug  5  2016 cldr.properties~
-rw-r--r--  1 surveytool cldr     1245 May 17 17:26 cldr.properties___
-rw-r--r--  1 surveytool cldr      726 May 28 16:31 cldr.properties.backup
-rw-r--r--  1 surveytool cldr       18 Aug  9 12:42 currev.properties
-rw-r-----  1 surveytool cldr 60988288 Sep 28 14:27 exception.log
-rw-r-----  1 surveytool cldr  1248692 Sep 22 06:24 exception.log-20180922.gz
-rw-r-----  1 surveytool cldr  1261518 Sep 23 06:25 exception.log-20180923.gz
-rw-r-----  1 surveytool cldr  1302120 Sep 24 06:25 exception.log-20180924.gz
-rw-r-----  1 surveytool cldr  1191401 Sep 25 06:22 exception.log-20180925.gz
-rw-r-----  1 surveytool cldr  1016090 Sep 26 06:24 exception.log-20180926.gz
-rw-r-----  1 surveytool cldr  1912763 Sep 27 06:24 exception.log-20180927.gz
-rw-r-----  1 surveytool cldr  1969220 Sep 28 06:24 exception.log-20180928.gz
drwxr-xr-x 10 surveytool cldr     4096 Jul 13  2016 vetdata

comment:17 Changed 7 weeks ago by tbishop

  • type changed from unknown to surveytool
  • Component changed from unknown to survey
  • Milestone changed from 34 to 35

comment:18 Changed 6 weeks ago by tbishop

76% full this morning!

cldradmin@st:~$ df -hT /home
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/sda1      ext4   29G   21G  6.9G  76% /
cldradmin@st:~$ ls -l /home/surveytool/tomcat/logs
total 9607836
-rw-r--r-- 1 surveytool cldr 160025585 Sep 10 23:59 catalina.2018-09-10.log
-rw-r--r-- 1 surveytool cldr 255328233 Sep 11 23:59 catalina.2018-09-11.log
-rw-r--r-- 1 surveytool cldr 235939801 Sep 12 23:59 catalina.2018-09-12.log
-rw-r--r-- 1 surveytool cldr 245722885 Sep 13 23:59 catalina.2018-09-13.log
-rw-r--r-- 1 surveytool cldr 240467148 Sep 14 23:59 catalina.2018-09-14.log
-rw-r--r-- 1 surveytool cldr 157950113 Sep 15 23:59 catalina.2018-09-15.log
-rw-r--r-- 1 surveytool cldr 217225271 Sep 16 23:59 catalina.2018-09-16.log
-rw-r--r-- 1 surveytool cldr 204271762 Sep 17 23:59 catalina.2018-09-17.log
-rw-r--r-- 1 surveytool cldr 174246018 Sep 18 23:59 catalina.2018-09-18.log
-rw-r--r-- 1 surveytool cldr 167353840 Sep 19 23:59 catalina.2018-09-19.log
-rw-r--r-- 1 surveytool cldr 174607483 Sep 20 23:59 catalina.2018-09-20.log
-rw-r--r-- 1 surveytool cldr 144691558 Sep 21 23:59 catalina.2018-09-21.log
-rw-r--r-- 1 surveytool cldr 117852532 Sep 22 23:59 catalina.2018-09-22.log
-rw-r--r-- 1 surveytool cldr 121056072 Sep 23 23:59 catalina.2018-09-23.log
-rw-r--r-- 1 surveytool cldr 129983594 Sep 24 23:58 catalina.2018-09-24.log
-rw-r--r-- 1 surveytool cldr  93109608 Sep 25 23:57 catalina.2018-09-25.log
-rw-r--r-- 1 surveytool cldr 165671991 Sep 26 23:59 catalina.2018-09-26.log
-rw-r--r-- 1 surveytool cldr 184709846 Sep 27 23:59 catalina.2018-09-27.log
-rw-r--r-- 1 surveytool cldr 194581453 Sep 28 23:59 catalina.2018-09-28.log
-rw-r--r-- 1 surveytool cldr 264138678 Sep 29 23:59 catalina.2018-09-29.log
-rw-r--r-- 1 surveytool cldr 243734639 Sep 30 23:59 catalina.2018-09-30.log
-rw-r--r-- 1 surveytool cldr 228545378 Oct  1 23:59 catalina.2018-10-01.log
-rw-r--r-- 1 surveytool cldr 236353822 Oct  2 23:59 catalina.2018-10-02.log
-rw-r--r-- 1 surveytool cldr 269513843 Oct  3 23:59 catalina.2018-10-03.log
-rw-r--r-- 1 surveytool cldr 268633150 Oct  4 23:59 catalina.2018-10-04.log
-rw-r--r-- 1 surveytool cldr 268248106 Oct  5 23:59 catalina.2018-10-05.log
-rw-r--r-- 1 surveytool cldr 278627866 Oct  6 23:59 catalina.2018-10-06.log
-rw-r--r-- 1 surveytool cldr 276998687 Oct  7 23:59 catalina.2018-10-07.log
-rw-r--r-- 1 surveytool cldr 273099927 Oct  8 23:59 catalina.2018-10-08.log
-rw-r--r-- 1 surveytool cldr 276540509 Oct  9 23:59 catalina.2018-10-09.log
-rw-r--r-- 1 surveytool cldr 148199014 Oct 10 13:02 catalina.2018-10-10.log
-rw-r--r-- 1 surveytool cldr  99765488 Oct 10 13:02 catalina.out
-rw-r--r-- 1 surveytool cldr   4259467 Oct  4 06:24 catalina.out-20181004.gz
-rw-r--r-- 1 surveytool cldr   4389917 Oct  5 06:24 catalina.out-20181005.gz
-rw-r--r-- 1 surveytool cldr   4244733 Oct  6 06:24 catalina.out-20181006.gz
-rw-r--r-- 1 surveytool cldr   4363471 Oct  7 06:24 catalina.out-20181007.gz
-rw-r--r-- 1 surveytool cldr   4340860 Oct  8 06:24 catalina.out-20181008.gz
-rw-r--r-- 1 surveytool cldr   4350366 Oct  9 06:25 catalina.out-20181009.gz
-rw-r--r-- 1 surveytool cldr   4519276 Oct 10 06:25 catalina.out-20181010.gz
-rw-r--r-- 1 surveytool cldr         0 Aug  6 13:31 host-manager.2018-08-06.log
-rw-r--r-- 1 surveytool cldr         0 Aug  8 21:13 host-manager.2018-08-08.log
-rw-r--r-- 1 surveytool cldr         0 Aug  9 01:32 host-manager.2018-08-09.log
-rw-r--r-- 1 surveytool cldr         0 Aug 10 15:24 host-manager.2018-08-10.log
-rw-r--r-- 1 surveytool cldr  75761153 Aug 30 23:59 localhost.2018-08-30.log
-rw-r--r-- 1 surveytool cldr  67640926 Aug 31 23:59 localhost.2018-08-31.log
-rw-r--r-- 1 surveytool cldr  72388416 Sep  1 23:59 localhost.2018-09-01.log
-rw-r--r-- 1 surveytool cldr  65834995 Sep  2 23:59 localhost.2018-09-02.log
-rw-r--r-- 1 surveytool cldr  47227181 Sep  3 23:59 localhost.2018-09-03.log
-rw-r--r-- 1 surveytool cldr  72419638 Sep  4 23:59 localhost.2018-09-04.log
-rw-r--r-- 1 surveytool cldr  71749103 Sep  5 23:59 localhost.2018-09-05.log
-rw-r--r-- 1 surveytool cldr  74147732 Sep  6 23:59 localhost.2018-09-06.log
-rw-r--r-- 1 surveytool cldr  70332350 Sep  7 23:59 localhost.2018-09-07.log
-rw-r--r-- 1 surveytool cldr  69681420 Sep  8 23:59 localhost.2018-09-08.log
-rw-r--r-- 1 surveytool cldr  66702569 Sep  9 23:59 localhost.2018-09-09.log
-rw-r--r-- 1 surveytool cldr  45415861 Sep 10 23:59 localhost.2018-09-10.log
-rw-r--r-- 1 surveytool cldr  72483650 Sep 11 23:59 localhost.2018-09-11.log
-rw-r--r-- 1 surveytool cldr  66982156 Sep 12 23:59 localhost.2018-09-12.log
-rw-r--r-- 1 surveytool cldr  69758690 Sep 13 23:59 localhost.2018-09-13.log
-rw-r--r-- 1 surveytool cldr  68268025 Sep 14 23:59 localhost.2018-09-14.log
-rw-r--r-- 1 surveytool cldr  44840805 Sep 15 23:59 localhost.2018-09-15.log
-rw-r--r-- 1 surveytool cldr  61668686 Sep 16 23:59 localhost.2018-09-16.log
-rw-r--r-- 1 surveytool cldr  57983036 Sep 17 23:59 localhost.2018-09-17.log
-rw-r--r-- 1 surveytool cldr  49466806 Sep 18 23:59 localhost.2018-09-18.log
-rw-r--r-- 1 surveytool cldr  47510830 Sep 19 23:59 localhost.2018-09-19.log
-rw-r--r-- 1 surveytool cldr  49546593 Sep 20 23:59 localhost.2018-09-20.log
-rw-r--r-- 1 surveytool cldr  41074719 Sep 21 23:59 localhost.2018-09-21.log
-rw-r--r-- 1 surveytool cldr  33454394 Sep 22 23:59 localhost.2018-09-22.log
-rw-r--r-- 1 surveytool cldr  34363911 Sep 23 23:59 localhost.2018-09-23.log
-rw-r--r-- 1 surveytool cldr  36887587 Sep 24 23:58 localhost.2018-09-24.log
-rw-r--r-- 1 surveytool cldr  26426960 Sep 25 23:57 localhost.2018-09-25.log
-rw-r--r-- 1 surveytool cldr  47014247 Sep 26 23:59 localhost.2018-09-26.log
-rw-r--r-- 1 surveytool cldr  52430756 Sep 27 23:59 localhost.2018-09-27.log
-rw-r--r-- 1 surveytool cldr  55211853 Sep 28 23:59 localhost.2018-09-28.log
-rw-r--r-- 1 surveytool cldr  74960191 Sep 29 23:59 localhost.2018-09-29.log
-rw-r--r-- 1 surveytool cldr  69187517 Sep 30 23:59 localhost.2018-09-30.log
-rw-r--r-- 1 surveytool cldr  64872085 Oct  1 23:59 localhost.2018-10-01.log
-rw-r--r-- 1 surveytool cldr  67079428 Oct  2 23:59 localhost.2018-10-02.log
-rw-r--r-- 1 surveytool cldr  76511004 Oct  3 23:59 localhost.2018-10-03.log
-rw-r--r-- 1 surveytool cldr  76254756 Oct  4 23:59 localhost.2018-10-04.log
-rw-r--r-- 1 surveytool cldr  76143587 Oct  5 23:59 localhost.2018-10-05.log
-rw-r--r-- 1 surveytool cldr  79091272 Oct  6 23:59 localhost.2018-10-06.log
-rw-r--r-- 1 surveytool cldr  78628773 Oct  7 23:59 localhost.2018-10-07.log
-rw-r--r-- 1 surveytool cldr  77506474 Oct  8 23:59 localhost.2018-10-08.log
-rw-r--r-- 1 surveytool cldr  78275281 Oct  9 23:59 localhost.2018-10-09.log
-rw-r--r-- 1 surveytool cldr  42069126 Oct 10 13:02 localhost.2018-10-10.log
-rw-r--r-- 1 surveytool cldr   3882151 Sep  1 23:59 localhost_access_log.2018-09-01.txt
-rw-r--r-- 1 surveytool cldr   3564490 Sep  2 23:59 localhost_access_log.2018-09-02.txt
-rw-r--r-- 1 surveytool cldr   3710090 Sep  3 23:59 localhost_access_log.2018-09-03.txt
-rw-r--r-- 1 surveytool cldr   4501578 Sep  5 00:00 localhost_access_log.2018-09-04.txt
-rw-r--r-- 1 surveytool cldr   5364117 Sep  5 23:59 localhost_access_log.2018-09-05.txt
-rw-r--r-- 1 surveytool cldr   5768738 Sep  6 23:59 localhost_access_log.2018-09-06.txt
-rw-r--r-- 1 surveytool cldr   5173747 Sep  8 00:00 localhost_access_log.2018-09-07.txt
-rw-r--r-- 1 surveytool cldr   3827063 Sep  9 00:00 localhost_access_log.2018-09-08.txt
-rw-r--r-- 1 surveytool cldr   3767167 Sep  9 23:59 localhost_access_log.2018-09-09.txt
-rw-r--r-- 1 surveytool cldr   4439002 Sep 11 00:00 localhost_access_log.2018-09-10.txt
-rw-r--r-- 1 surveytool cldr   6119746 Sep 11 23:59 localhost_access_log.2018-09-11.txt
-rw-r--r-- 1 surveytool cldr   5494507 Sep 12 23:59 localhost_access_log.2018-09-12.txt
-rw-r--r-- 1 surveytool cldr   6560500 Sep 14 00:00 localhost_access_log.2018-09-13.txt
-rw-r--r-- 1 surveytool cldr   5732625 Sep 15 00:00 localhost_access_log.2018-09-14.txt
-rw-r--r-- 1 surveytool cldr   4496193 Sep 16 00:00 localhost_access_log.2018-09-15.txt
-rw-r--r-- 1 surveytool cldr   5055806 Sep 17 00:00 localhost_access_log.2018-09-16.txt
-rw-r--r-- 1 surveytool cldr   5545152 Sep 18 00:00 localhost_access_log.2018-09-17.txt
-rw-r--r-- 1 surveytool cldr   4705100 Sep 19 00:00 localhost_access_log.2018-09-18.txt
-rw-r--r-- 1 surveytool cldr   6262742 Sep 20 00:00 localhost_access_log.2018-09-19.txt
-rw-r--r-- 1 surveytool cldr   5780158 Sep 21 00:00 localhost_access_log.2018-09-20.txt
-rw-r--r-- 1 surveytool cldr   5476740 Sep 22 00:00 localhost_access_log.2018-09-21.txt
-rw-r--r-- 1 surveytool cldr   4174759 Sep 23 00:00 localhost_access_log.2018-09-22.txt
-rw-r--r-- 1 surveytool cldr   4692402 Sep 23 23:59 localhost_access_log.2018-09-23.txt
-rw-r--r-- 1 surveytool cldr   6629973 Sep 25 00:00 localhost_access_log.2018-09-24.txt
-rw-r--r-- 1 surveytool cldr   4287355 Sep 26 00:00 localhost_access_log.2018-09-25.txt
-rw-r--r-- 1 surveytool cldr   5026854 Sep 27 00:00 localhost_access_log.2018-09-26.txt
-rw-r--r-- 1 surveytool cldr   4361071 Sep 28 00:00 localhost_access_log.2018-09-27.txt
-rw-r--r-- 1 surveytool cldr   4541001 Sep 29 00:00 localhost_access_log.2018-09-28.txt
-rw-r--r-- 1 surveytool cldr   4156589 Sep 29 23:59 localhost_access_log.2018-09-29.txt
-rw-r--r-- 1 surveytool cldr   3896110 Sep 30 23:59 localhost_access_log.2018-09-30.txt
-rw-r--r-- 1 surveytool cldr   5246676 Oct  1 23:59 localhost_access_log.2018-10-01.txt
-rw-r--r-- 1 surveytool cldr   4470322 Oct  3 00:00 localhost_access_log.2018-10-02.txt
-rw-r--r-- 1 surveytool cldr   4428404 Oct  4 00:00 localhost_access_log.2018-10-03.txt
-rw-r--r-- 1 surveytool cldr   4275176 Oct  4 23:59 localhost_access_log.2018-10-04.txt
-rw-r--r-- 1 surveytool cldr   4301082 Oct  6 00:00 localhost_access_log.2018-10-05.txt
-rw-r--r-- 1 surveytool cldr   3954275 Oct  6 23:59 localhost_access_log.2018-10-06.txt
-rw-r--r-- 1 surveytool cldr   3788739 Oct  8 00:00 localhost_access_log.2018-10-07.txt
-rw-r--r-- 1 surveytool cldr   4052767 Oct  8 23:59 localhost_access_log.2018-10-08.txt
-rw-r--r-- 1 surveytool cldr   5544284 Oct  9 23:59 localhost_access_log.2018-10-09.txt
-rw-r--r-- 1 surveytool cldr   3271766 Oct 10 13:02 localhost_access_log.2018-10-10.txt
-rw-r--r-- 1 surveytool cldr         0 Aug  6 13:31 manager.2018-08-06.log
-rw-r--r-- 1 surveytool cldr         0 Aug  8 21:13 manager.2018-08-08.log
-rw-r--r-- 1 surveytool cldr         0 Aug  9 01:32 manager.2018-08-09.log
-rw-r--r-- 1 surveytool cldr         0 Aug 10 15:24 manager.2018-08-10.log
-rw-r--r-- 1 root       root     78105 Aug  9 12:42 tomcat.log
-rw-r--r-- 1 root       root  52437838 Jul 30 11:26 tomcat.log.1
-rw-r--r-- 1 root       root  52429677 Jul 28 12:41 tomcat.log.10
-rw-r--r-- 1 root       root  52430148 Jul 30 06:26 tomcat.log.2
-rw-r--r-- 1 root       root  52428923 Jul 30 01:54 tomcat.log.3
-rw-r--r-- 1 root       root  52435900 Jul 29 20:53 tomcat.log.4
-rw-r--r-- 1 root       root  52437562 Jul 29 15:10 tomcat.log.5
-rw-r--r-- 1 root       root  52438700 Jul 29 09:46 tomcat.log.6
-rw-r--r-- 1 root       root  52429821 Jul 29 04:56 tomcat.log.7
-rw-r--r-- 1 root       root  52436076 Jul 28 23:27 tomcat.log.8
-rw-r--r-- 1 root       root  52431124 Jul 28 18:02 tomcat.log.9
cldradmin@st:~$ ls -l /home/surveytool/tomcat/cldr
total 141276
drwxr-xr-x  4 surveytool cldr     4096 Dec 16  2015 28
drwxr-xr-x  4 surveytool cldr     4096 Mar 30  2016 29
drwxr-xr-x  4 surveytool cldr     4096 Oct 20  2016 30
drwxr-xr-x  4 surveytool cldr     4096 Mar 31  2017 31
drwxr-xr-x  4 surveytool cldr     4096 Nov  9  2017 32
drwxr-xr-x  4 surveytool cldr     4096 Apr 26 19:55 33
-rw-r--r--  1 surveytool cldr      656 Dec 16  2015 admin.html
drwxr-xr-x  2 surveytool cldr     4096 Sep  7  2017 cache
drwxr-xr-x  5 surveytool cldr     4096 Dec 16  2015 cldrdb
-rw-r--r--  1 surveytool cldr 49903022 Oct  8 20:26 cldrmail.log
-rw-r--r--  1 surveytool cldr      703 Aug 10 15:23 cldr.properties
-rw-r--r--  1 surveytool cldr      871 Aug  5  2016 cldr.properties~
-rw-r--r--  1 surveytool cldr     1245 May 17 17:26 cldr.properties___
-rw-r--r--  1 surveytool cldr      726 May 28 16:31 cldr.properties.backup
-rw-r--r--  1 surveytool cldr       18 Aug  9 12:42 currev.properties
-rw-r-----  1 surveytool cldr 74999339 Oct 10 13:02 exception.log
-rw-r-----  1 surveytool cldr  2749395 Oct  4 06:24 exception.log-20181004.gz
-rw-r-----  1 surveytool cldr  2785755 Oct  5 06:24 exception.log-20181005.gz
-rw-r-----  1 surveytool cldr  2768483 Oct  6 06:24 exception.log-20181006.gz
-rw-r-----  1 surveytool cldr  2851039 Oct  7 06:24 exception.log-20181007.gz
-rw-r-----  1 surveytool cldr  2840226 Oct  8 06:24 exception.log-20181008.gz
-rw-r-----  1 surveytool cldr  2817910 Oct  9 06:25 exception.log-20181009.gz
-rw-r-----  1 surveytool cldr  2816004 Oct 10 06:25 exception.log-20181010.gz
drwxr-xr-x 10 surveytool cldr     4096 Jul 13  2016 vetdata

comment:19 Changed 6 weeks ago by tbishop

I used rsync to copy these folders to an external volume connected to my macBook:

/home/surveytool/tomcat/logs
/home/surveytool/tomcat/cldr

I did this non-recursively, since especially cldr contains subdirectories with a huge amount of non-log data.

Commands run from my own machine:

rsync -hvz cldradmin@st.unicode.org:/home/surveytool/tomcat/logs/* /Volumes/WDMyBookSegundo/backup_surveytool/logs

rsync -hvz cldradmin@st.unicode.org:/home/surveytool/tomcat/cldr/* /Volumes/WDMyBookSegundo/backup_surveytool/cldr

Then I deleted some of the older logs from the server as follows:

surveytool@st:~/tomcat/logs$ rm localhost_access_log.2018-09*
surveytool@st:~/tomcat/logs$ rm localhost.2018-09*
surveytool@st:~/tomcat/logs$ rm localhost.2018-08*
surveytool@st:~/tomcat/logs$ rm catalina.2018-09*

surveytool@st:~/tomcat/logs$ df -hT /home
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/sda1      ext4   29G   16G   13G  56% /

That reduced the usage to 56%.

comment:20 Changed 3 weeks ago by tbishop

Again I did rsync tomcat/logs and tomcat/cldr to an external volume connected to my macBook. Then I deleted some older logs.

Before:

cldradmin@st:~$ df -hT /home
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/sda1      ext4   29G   19G  8.9G  68% /

After:

surveytool@st:~/tomcat/cldr$ df -hT /home
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/sda1      ext4   29G   13G   15G  47% /
View

Add a comment

Modify Ticket

Action
as accepted
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.