DaemonForums  

Go Back   DaemonForums > FreeBSD > FreeBSD General

FreeBSD General Other questions regarding FreeBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 13th February 2009
goertzenator goertzenator is offline
New User
 
Join Date: Feb 2009
Location: Winnipeg
Posts: 5
Default apache: data stuck in socket?

I've got apache running in a new jail, and it is unable to respond to requests. It looks like the http request is not getting to apache. I have another jail running asterisk which works fine (although it is much older), so I am stumped and could use some hints.

Thanks,
Dan.


On to the details:

The jail is 192.168.1.81 "bug" (I want to setup bugzilla on it)


From outside the jail:

Code:
[root@boondock ~]# uname -a
FreeBSD boondock.luey.org 7.1-RELEASE-p2 FreeBSD 7.1-RELEASE-p2 #2: Thu Feb 12 15:12:18 CST 2009     root@boondock.luey.org:/usr/obj/usr/src/sys/GENERIC  amd64
Code:
[root@boondock ~]# tcpdump -A host 192.168.1.81
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on net0, link-type EN10MB (Ethernet), capture size 96 bytes
10:35:01.116146 IP 192.168.1.105.40398 > 192.168.1.81.http: S 518683053:518683053(0) win 5840 <mss 1460,sackOK,timestamp 80851933 0,nop,wscale 7>
E..<.p@.@..@...i...Q...P..y....................
............
10:35:01.116156 IP 192.168.1.81.http > 192.168.1.105.40398: S 270420199:270420199(0) ack 518683054 win 65535 <mss 1460,nop,wscale 3,sackOK,timestamp 2592150776 80851933>
E..<..@.@......Q...i.P....H...y......9.............
........
10:35:01.116401 IP 192.168.1.105.40398 > 192.168.1.81.http: . ack 1 win 46 <nop,nop,timestamp 80851933 2592150776>
E..4.q@.@..G...i...Q...P..y...H............
........
10:35:01.116410 IP 192.168.1.105.40398 > 192.168.1.81.http: P 1:377(376) ack 1 win 46 <nop,nop,timestamp 80851934 2592150776>
E....r@.@......i...Q...P..y...H......s.....
........GET / HTTP/1.1
Host: bug
Use
10:35:01.216382 IP 192.168.1.81.http > 192.168.1.105.40398: . ack 377 win 8279 <nop,nop,timestamp 2592150877 80851934>
E..4..@.@......Q...i.P....H...{&.. W.1.....
...]....

[hangs forever.  Firefox on 192.168.1.105 waits forever.]
From inside the jail:

Code:
[root@bug /]# uname -a
FreeBSD bug.luey.org 7.1-RELEASE-p2 FreeBSD 7.1-RELEASE-p2 #2: Thu Feb 12 15:12:18 CST 2009     root@boondock.luey.org:/usr/obj/usr/src/sys/GENERIC  amd64
[
Code:
root@bug /]# netstat
netstat: kvm not available: /dev/mem: No such file or directory
Active Internet connections
Proto Recv-Q Send-Q  Local Address          Foreign Address        (state)
tcp4     376      0 bug.http               192.168.1.105.40398    ESTABLISHED
tcp4     376      0 bug.http               192.168.1.105.45122    CLOSE_WAIT
tcp4     376      0 bug.http               192.168.1.105.41796    CLOSE_WAIT
tcp4     376      0 bug.http               192.168.1.105.32904    CLOSE_WAIT
udp4       0      0 bug.syslog             *.*
Active UNIX domain sockets
Address  Type   Recv-Q Send-Q    Inode     Conn     Refs  Nextref Addr
<snip>
"top" shows that httpd's state is "select".


Code:
[root@bug /]# tail /var/log/httpd-error.log
[Thu Feb 12 22:38:10 2009] [info] Init: Seeding PRNG with 144 bytes of entropy
[Thu Feb 12 22:38:10 2009] [info] Init: Generating temporary RSA private keys (512/1024 bits)
[Thu Feb 12 22:38:11 2009] [info] Init: Generating temporary DH parameters (512/1024 bits)
[Thu Feb 12 22:38:11 2009] [info] Init: Initializing (virtual) servers for SSL
[Thu Feb 12 22:38:11 2009] [info] mod_ssl/2.2.11 compiled against Server: Apache/2.2.11, Library: OpenSSL/0.9.8e
[Thu Feb 12 22:38:11 2009] [notice] Digest: generating secret for digest authentication ...
[Thu Feb 12 22:38:11 2009] [notice] Digest: done
[Thu Feb 12 22:38:11 2009] [notice] Apache/2.2.11 (FreeBSD) mod_ssl/2.2.11 OpenSSL/0.9.8e DAV/2 configured -- resuming normal operations
[Thu Feb 12 22:38:11 2009] [info] Server built: Jan 21 2009 08:13:02
[Thu Feb 12 22:38:11 2009] [debug] peruser.c(2380): AcceptMutex: flock (default: flock)
Code:
[root@bug /]# tail /var/log/httpd-access.log
[root@bug /]#
Code:
[root@bug /]# pkg_info                                                                            
apache-peruser-2.2.11_2 Version 2.2.x of Apache web server with peruser MPM.  
<snip>
Reply With Quote
  #2   (View Single Post)  
Old 13th February 2009
anomie's Avatar
anomie anomie is offline
Local
 
Join Date: Apr 2008
Location: Texas
Posts: 445
Default

Some thoughts:
  • I am a little confused by the tcpdump output. It looks like it is showing SYN -> SYN -> ACK. (Should be SYN -> SYN-ACK -> ACK.)
  • It would probably be worthwhile to post the output of % sockstat -4l (instead of the netstat output you posted).
  • apache22-peruser-mpm is just a module, right? You must also have apache web server installed. (That's how it appears to me on freshports anyway.)

Anyway, some basic questions:
  • Is networking set up properly for the jail? Can you try a tcp connect scan from the jail to another host your network? e.g. % nc -zvw 1 host.here 22
  • Are you running any packet filtering firewall on the host?
  • Could you triple check to make sure httpd.conf is really logging to /var/log/httpd-access.log?
__________________
Kill your t.v.
Reply With Quote
  #3   (View Single Post)  
Old 13th February 2009
goertzenator goertzenator is offline
New User
 
Join Date: Feb 2009
Location: Winnipeg
Posts: 5
Default

Hi anomie, thanks for the reply.

1.
Code:
[root@bug ~]# sockstat -4l
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS
root     sendmail   1870  4  tcp4   192.168.1.81:25       *:*
root     httpd      1859  3  tcp4   192.168.1.81:80       *:*
root     syslogd    1793  6  udp4   192.168.1.81:514      *:*

(taken while firefox is trying to load a page from httpd)
2. The apache package description is "Apache web server with peruser MPM". This is what I got when I did "pkg_add -r apache22". Apache is definitely there.


3.
Code:
[root@bug /]# nc -zvw 1 192.168.1.105 22
Connection to 192.168.1.105 22 port [tcp/ssh] succeeded!
4. I had some old rules up and pf was enabled. I turned them off but it did not help.

5. Access log config is okay. Only change I made from default was to set LogLevel to debug. httpd.conf fragment below

Code:
#
# ErrorLog: The location of the error log file.
# If you do not specify an ErrorLog directive within a <VirtualHost>
# container, error messages relating to that virtual host will be
# logged here.  If you *do* define an error logfile for a <VirtualHost>
# container, that host's errors will be logged there and not here.
#
ErrorLog "/var/log/httpd-error.log"

#
# LogLevel: Control the number of messages logged to the error_log.
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
#

#LogLevel warn
LogLevel debug

<IfModule log_config_module>
    #
    # The following directives define some format nicknames for use with
    # a CustomLog directive (see below).
    #
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %b" common

    <IfModule logio_module>
      # You need to enable mod_logio.c to use %I and %O
      LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
    </IfModule>

    #
    # The location and format of the access logfile (Common Logfile Format).
    # If you do not define any access logfiles within a <VirtualHost>
    # container, they will be logged here.  Contrariwise, if you *do*
    # define per-<VirtualHost> access logfiles, transactions will be
    # logged therein and *not* in this file.
    #
    #CustomLog "/var/log/httpd-access.log" common

    #
    # If you prefer a logfile with access, agent, and referer information
    # (Combined Logfile Format) you can use the following directive.
    #
    CustomLog "/var/log/httpd-access.log" combined
</IfModule>
I've made no other changes to httpd.conf... I just want to see the "It works!" page.
Reply With Quote
  #4   (View Single Post)  
Old 13th February 2009
goertzenator goertzenator is offline
New User
 
Join Date: Feb 2009
Location: Winnipeg
Posts: 5
Default

The second packet is a SYN-ACK. On my browser I have to scroll the bar to the right a bit to see the ACK, I bet yours is the same.

Dan.
Reply With Quote
  #5   (View Single Post)  
Old 13th February 2009
anomie's Avatar
anomie anomie is offline
Local
 
Join Date: Apr 2008
Location: Texas
Posts: 445
Default

Odd. That all looks fine.

Quote:
Originally Posted by goertzenator
The second packet is a SYN-ACK. On my browser I have to scroll the bar to the right a bit to see the ACK, I bet yours is the same.
Whoops, I missed it. You are right.

OK, so next let's try to eliminate apache from the equation. Shut down apache in the jail, and then within the jail run: # nc -l -p 80

Then from your workstation (i.e. same workstation that you are testing with a web browser), run: % nc jail.ip.here 80

Does that connect? If so, start typing text in the workstation terminal. That text should be echoed to the listening nc server within the jail. All that work or not?
__________________
Kill your t.v.
Reply With Quote
  #6   (View Single Post)  
Old 13th February 2009
goertzenator goertzenator is offline
New User
 
Join Date: Feb 2009
Location: Winnipeg
Posts: 5
Default

That worked. No problem sending text in both directions. (I used nc -l 80 from the jail)
Reply With Quote
  #7   (View Single Post)  
Old 13th February 2009
anomie's Avatar
anomie anomie is offline
Local
 
Join Date: Apr 2008
Location: Texas
Posts: 445
Default

I'm not sure what to make of it at this point then. tcp connections can be be established to and from the jail. Yet apache is logging no access attempts, which is suspicious. I am (mostly) ruling out an apache configuration problem because you say you have changed only LogLevel following the package install; and in any case we should see chatter in at least the error log.

I think what I'd do next is:
  • check /var/log/messages for NIC complaints;
  • try viewing the welcome page from a different workstation (and different browser - e.g. lynx);
  • and if all else fails, try running a different service - e.g. ftpd - within the jail (does it work or not?)

Finally, it doesn't seem like a network misconfiguration is the culprit, but it would be a good idea to post the output of % ifconfig from the host, and also your jail-related directives in rc.conf on the host.
__________________
Kill your t.v.
Reply With Quote
  #8   (View Single Post)  
Old 14th February 2009
BSDfan666 BSDfan666 is offline
Real Name: N/A, this is the interweb.
Banned
 
Join Date: Apr 2008
Location: Ontario, Canada
Posts: 2,223
Default

I'm a little out of my area of expertise, but is it possible that apache is binding to the wrong IP address?

Can you post your full configuration?
Reply With Quote
  #9   (View Single Post)  
Old 16th February 2009
goertzenator goertzenator is offline
New User
 
Join Date: Feb 2009
Location: Winnipeg
Posts: 5
Default

I built up another jail (bug2), added some basic tools that I like (bash, nano, rsync), and then added just apache22. What do you know, it worked!

I tried yet another jail (bug3) in an attempt to replicated my original steps in the non-working jail (bug). I added my basic tools, then bugzilla, then mysql51-server, then apache22. It just worked. Hmmmmmm.

bug2 and bug3 were definitely built with a fresh RELENG_7_1 on a host that is definitely also RELENG_7_1. bug was built with a different copy of RELENG_7_1 although I have limited experience with csup so it may have in fact been older. The host was definitely older than RELENG_7_1 at the time.

So I guess I have things going now, and I am inclined to write off the original non-working apache jail as something I screwed up due to mismatched versions.

I apologize for the wild goose chase, but the advice on this forum pushed me in the right direction. Thanks!

Dan.

Last edited by goertzenator; 16th February 2009 at 04:04 PM. Reason: wrong version label, changed to RELENG_7_1
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
FreeBSD 7.2@amd64 atheros problem ath0: stuck beacon; resetting (bmiss count 4) asmo FreeBSD General 0 6th June 2009 11:02 AM
Socket A Semprons Sonya General Hardware 7 17th January 2009 04:32 AM
data recovery. LateNiteTV FreeBSD General 8 29th August 2008 08:11 PM
Apache data transfer limit cajunman4life General software and network 5 7th June 2008 05:13 PM
Java socket programming question 18Googol2 Programming 3 3rd May 2008 11:50 AM


All times are GMT. The time now is 03:44 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick